It might have been filled with mineral oil, those external enclosures often setup that way so that the enclosure is less extreme to manufacture. Not sure if that would work for camera lenses though unless those were also filled.
As someone who was on the product team for a 6000msw video camera, it's probably not filled with mineral oil. I doubt anyone makes subsea camera bodies/optics completely from scratch, and off-the-shelf units are not designed for pressure. In ours we used Sony camera internals, the enclosure was atmospheric, filled with dry nitrogen to reduce condensation, and the sapphire lens was designed to resist 12500psi and reduce distortion from the air-->sapphire-->seawater interface.
Hmm ok that would have explained why the SD card wasn't damaged if it were because when the vessel is filled with mineral oil it wouldn't have imploded like the main body of the craft.
Here; the title focusing on the price is implying that the cheap SD card survived ocean floor environment alone. A surprising amount of stress for its price.
Instead, a pressure-proof deep sea camera module was found at the wreckage site. It’s less interesting that an expensive thing rated for ocean depths was intact at ocean depths.
Its like “missing child found after 4 days in Alaska temperatures!”
gasp! How did they survive!
“The child was on holiday in their grandparents’ holiday log cabin, with their grandparents, a log fire, food, water..”
Oh. Clickbait. Hiding the boring bit to make the story appear more of a tease.
I noticed a pattern a few months ago in my phone's newsbait feed of headlines in the format "[Large familiar company] to close 500 stores on [date]" — and then below the fold "because it's [Independence Day, Rosh Hashanah, etc]" or "because they are moving to summer hours" or whatever.
> Here; the title focusing on the price is implying that the cheap SD card survived ocean floor environment alone. A surprising amount of stress for its price.
I certainly did not read that implication into the title, so it's entirely possible that the author didn't mean it.
The manufacturer didn’t even know encryption was enabled, because as long as the camera was working, it would just provide all files over USB without any encryption.
It was basically enabled by accident, and the only thing it prevented was recovery of files directly from the SD card when the camera was damaged.
I think it depends. Encrypted filesystems typically encrypt contents of each file separately - that way you don't need to read / write the whole disk to read it write any individual file contents. Of course that means metadata may be in plain text or may be separately encrypted - again possibly folder by folder instead of all metadata at once. Exact details would vary with different file system encryption schemes.
Whereas if you image the disk and encrypt the image properly, that gives you all the great confidentially guarantees but no random access.
> Encrypted filesystems typically encrypt contents of each file separately - that way you don't need to read / write the whole disk to read it write any individual file contents.
Ah, that's not true of "full disk encryption". It usually encrypts the disk blocks.
File-based encryption is stronger; you can use different protection classes on different files, you can use authenticated encryption, etc. iOS does it this way and I assume other systems have caught up, but don't know any in particular.
Most FDE systems are not authenticated so you would only lose one block (16 bytes for AES). Can this be bad? Yeah, but it's not that bad for data recovery.
Most unauthenticated encryption modes only mess up a few bits of a block, sometimes the following block too. A few only flip the exact bit in the plaintext.
Citation needed. It might be slightly easier, but most cases where you can get part of the camera, you can get the whole camera. This isn't a little point-and-click with a handy spring-loaded slot either.
Yeah but the Camera's owner is much more likely to notice "my camera is missing" than "the SD card is blank for some reason... the SD card must have failed"
EDIT: The linked PDF has a photo, the camera literally opens up to access the SD card.
The camera's (former) owner may very well notice, but that will have little effect. It's much more common that cameras (security, photography, phones) get stolen with cards inside, rather than someone extracting the card and leaving the camera.
Worth mentioning that I would immediately know if a different SD card was in my camera the moment I turned it on or ejected the card. If somebody knew to buy the same exact model and storage size that would be truly impressive.
Industrial espionage is far far more often done by hard work then being clever. Checking the SD cards you use and buying matching ones before executing a swap isn't noteworthy.
If you look through my case of SD cards I have all sorts of sizes and makes/models. I also have a procedure for dumping and formatting, and someone who is handling this most likely would as well. The moment the storage on screen looks funny or I see the card I don’t recognize, I’ll notice.
I’m not saying it’s impossible or I’m somehow immune to being tricked, but you would be surprised how easy it is to leave evidence of tampering through the simple act of trying to take an SD card, whether you swap it out or not. And people like me who handle them regularly would likely notice it in a split second. Maybe we won’t even catch the perpetrator, but it would not be written off as “oh interesting I guess it didn’t record,” if for no other reason then I would go straight into CYA mode and start looking for any hint that I didn’t make a mistake.
At that point if you have a basic security SoP in place and adhere to it you can start auditing.
Anywho again nothing is airtight and a determined individual could of course get past me. But you would be surprised how many hurdles they have to get over, and it certainly wouldn’t go unnoticed and be brushed off as a read/write quirk, that’s really all I’m saying
If you read the article, the SD card was placed there by the camera manufacturer and then the device was sealed so it would withstand pressure, and then sold to divers. Blame the camera manufacturer's engineers.
Seems like the SD card of all things performed just fine, so it hardly seems like the weak point.
They are surprisingly resilient! There was a blog post a couple of years ago by someone who found a digital camera in the sea which must have been in the water for months. The author could look at the photos to see if they can find the original owner of the camera.
> No deep-sea shenanigans around the Titanic wreck were revealed. Manley explains in his Twitter thread that “the camera had been configured to dump data onto an external storage device, so nothing was found from the accident dive.” Nothing particularly pertinent to the tragic accident, that is.
This is about camera hardware and how it survived. It provides no information or footage about the incident (in case you were looking for it like I was).
The System on Module board is an Inforce 6601 SOM. [0]
It uses a Qualcomm Snapdragon 820 and they provide prebuilt Ubuntu Linaro distros for it, preconfigured for the board.
The camera manufacturer likely just tossed it straight in as configured and thus didn't know how the full disk encryption was setup.
This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'.
See also: almost any specialty medical electronic device ever manufactured.
> This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'.
> See also: almost any specialty medical electronic device ever manufactured.
These are not design mistakes.
When building products in short runs and where the costs of part have little impact on your margins compared to R&D, it completely makes sense to go for a full computer rather than bother with embedded development where everything is more complicated. Medical also has to deal with certification which is a much more significant concern than saving on parts and will often reuse already certified components.
I'll admit I only watched a video on it not the report, but it had pictures reportedly redacted at manufacturer request. It showed a teensy 3 and some adafruit qwiic board in there. Obviously the real engineering is in the enclosure. Otherwise it could just be a webcam. But still, it's clearly not a very in depth electrical design. I'm all for SoMs if you can but they don't guarantee you the adventure of custom hardware bringing moving through all the software stacks and whatnot.
Usually (not always), something like a Teensy or a Pi Pico or an Arduino is treated like a development board for prototyping.
A person builds out their circuit using hardware they can solder/wire-up by hand on a workbench, maybe even with relatively-giant solderless breadboards, to prove the concept and the general design.
And a dev board can be great for spinning a few prototypes. It's quick to get started (code can begin being tested on-chip after just plugging in a USB cable), and to try different things and to make (and correct!) mistakes. (Blow up a Teensy? No worries; just grab another from the drawer, try not to make that same mistake again, and keep moving -- no esoteric soldering required)
But when the design is finished-enough and it becomes time to spin up custom-built PCBs for a final product that will be sold, a separate dev board like a Teensy tends to lose much of its initial charm.
Instead, it's more-typical just put the microcontroller IC plus whatever supporting hardware is necessary for the overall device's actual functions on the main board. Don't need USB, or an Ethernet PHY, an LED, a button, or a separate voltage regulator? Want more or less flash? When including the MCU on a board of one's own design instead of a kitchen-sink dev board, one is empowered to use exactly the parts that are required.
This can save a substantial amount of space and greatly improve the flexibility of the layout, while also improving mechanical and electrical robustness by having fewer connections between the MCU and the world around it. Plus, fewer parts tend to be less costly than more parts are.
(But again, it's not always done this way. This camera from the submarine is an example of one instance where the whole dev board was put inside of a finished product. Sometimes that's a good idea, and sometimes it isn't. I'm not attempting to suggest that it was or was not a good move in this instance.)
I suspect you're right about the quantities. In support of that notion, when looking closer at the (linked in another user's comment) PDF of the report, I can see that a lot of this camera's internal structure quite clearly appears to be the product of an FDM 3D printer. This suggests that quantities are low.
And I don't know when that camera was manufactured or designed.
But these days, it's possible to get even hobbyist-quantities of custom PCBs delivered with difficult-to-solder ICs installed from sources like JLCPCB.
(Depending on the features and functions wanted, it doesn't take a whole lot of extra parts to get an MCU to do its thing: There's not a ton of parts on a Teensy to begin with.)
Everything you said makes sense except you haven't explained why you can't just seal up a Teensy in an enclosure and sell it that way, except for "you're not supposed to do that". Are Teensies prone to random failure or something? Because if they just work and you're only selling <50 devices for extremely specialized nieche then I really don't see a problem with this?
Well, but it's not about whether someone can or cannot do something. Since you seem experienced with these devices I'm just asking if there's any technical reason why this might be a bad idea other than the fact that it just doesn't seem like a very professional thing to do. Like for example even though I'm far from an expert I know Raspberry Pis would be awful for any commercial application because they are notorious for killing their SD cards rendering the device useless.
The Teensy wasn’t engineered, tested, rated, or certified for any sort of continuous duty, let alone within a pressurized O2-enriched environment (assuming it was inside the vessel), especially not within deep sea Helium-enriched environments (that have been shown to break things like MEMS devices), and present unnecessary risks for an entirely inefficient choice (see: comment above, Teensy’s are ~$30-40/ea where small PCBs populated with the same circuit features can be had for under $20).
I’m probably not as qualified as the person you replied to, but that’s my intuition as someone with a passing familiarity with electronics engineering (I have an associates degree in EE).
People do whatever they want. It doesn't have to make sense.
Perhaps disturbingly: I even know of one bit of critical public safety communications infrastructure that is is expensive, low production volume, and has a Raspberry Pi 3 embedded inside. I won't name names because that's getting a little too close to home for my liking, but I was quite surprised to find this inside of a very nice waterproof box with chonky, expensive, olive-colored milspec connectors to connect it up to the outside world.
Which, well: Yeah. There's a ton of good reasons not to do that. But building a whole Linux system on a custom board using individual parts is hard, so it can make sense to buy someone else's work instead.
Except... that's what the CM3 is designed to provide, including on-board eMMC instead of an SD card. I'd not have been surprised at all if there was a CM3 in there, but there is instead an entire Pi 3.
But MCUs, like on the Teensy, aren't like that. They aren't hard to integrate on a custom board like the Broadcom SoC on a Pi 3 or CM3 is.
The primary purpose of an MCU is not to be stuffed onto a dev board like a Teensy, but instead to be stuffed onto the board inside of a microwave oven or an air fryer or a fancy remote control and be easy to interface with other things and to program.
It really doesn't take much to get them going: Some require external ROM or flash, but a lot of them have internal flash memory and only need power and programming pins wired up to let them run code and do whatever IO is needed within a system.
This camera already had at least one very custom board inside. It could have integrated the MCU, as well, instead of the kitchen-sink Teensy.
Doing so is not just style points; it's quite often easier, cheaper, and more flexible.
This allows a person to use all of the IO pins on the MCU to do stuff with, instead of just the functions that the designer of a dev kit decided to build out through whatever interfaces they decided to include.
They had no idea how their own product worked. They didn’t even know it used encrypted storage.
This was either outsourced or done by some junior engineer who was putting pieces together like it was another Raspberry Pi project that just needed to kind of work.
The longer I last in this world the more products I realize are the result of telling a few people who don't know what they are doing to "make it kind of work."
That’s my entire experience in embedded. Everything I get from other companies basically looks like an internship project right down to the pointer arguments with unspecified bounds on the function calls. One of the companies we bought hardware from keeps representing things are working when they only work on devices in the lab. Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
> Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
While I agree with your sentiment that there's a lot of poor software engineering in embedded space (especially in consumer-oriented novelty products, less so in established fields like industrial or telco), I can't but wonder what's wrong with Yocto? In my experience, it's quite the opposite: Yocto is the quickest path to get the firmware for a new device assembled, once you have climbed its pretty steep learning curve. I have built a few homebrew firmware build systems out of Debian and make/shell scripts (not my choice), you pretty quickly find yourself reinventing half of the stuff that Yocto does out of the box, but it's all bespoke, janky and hard to maintain. While with Yocto you just take the vendor's meta layer for BSP, put your application in another, and it bakes you a set of flashable images on the other end, complete with SDKs and other goodies for your dev workflow, reproducibly. It doesn't get significantly more sophisticated once you start to need kernel customizations, firmware updates with A/B partition layout, readonly rootfs, manage board- or customer-specific variants and other features that are very common in embedded systems but poorly or not at all supported in standard distros.
The problem isn’t OE per se but every vendor pulls in their own janky patches for everything and you find yourself fighting the build system to cut packages and reduce the filesystem size. It’s also really slow and it’s frankly massive, requiring tens or hundreds of gigabytes of stuff to yield less than a gigabyte of system image. And at the end of the day if a vendor breaks something from another vendor you will find yourself struggling to figure out why. Older versions of their documentation make statements that small teams should avoid Yocto for these reasons.
If you don’t already know anything about how to do these things in Linux Yocto is good for it. But if you try to swim against the current of the way the layer authors want you to do any of those things you will find it very challenging.
Take read-only root, for example. Usually it’s a one-line change in a config file plus an additional mount should you want an overlay in volatile memory. I don’t see how pulling in layers and config files does anything other than obscure how that works.
For a good example of what I hate about it, try building from Meta-Intel without graphics drivers.
I'm mostly in the software space but in the past few years I've been doing a lot more embedded stuff, and the trend I notice is that companies are making great hardware, and then completely ruining its usefulness with bad software and firmware. It's kind of mind blowing to me because I always considered software to be the easy part of making a product, compared to, you know, etching microscopic patterns onto sand to make magical transistors appear in just the right way to do the task you want.
In this context? Certainly. This is more examples of their using untested, uncertified equipment. Anyone who’s ever worked in O2-enriched pressurized environments should at least be instantly concerned about whether this device had been properly rated for fire prevention.
It's a solid piece of silicon encased in epoxy, so there's nothing really to get crushed. Contrast this to something like a cellphone that's made of hundreds of separate parts and has void space that will get crushed.
Are consumer grade cards really reliable though? Not so much against physical damage, but of data integrity over extended periods? "Industrial" SD cards can be 10 times or 100 times more expensive than consumer grade cards.
This comment made me wonder how much easier proximity fuzes would have been to develop in WW2 had they had transistors (or integrated circuits). I assume making modern solid state electronics 20,000g shock resistant is much easier than doing the same to vacuum tubes.
How would you do screen replacement? That is a common repair since people drop their phones and currently you can get your phone repaired by some teenager in a booth at the mall. If you fill the phone with epoxy, how are you detaching the screen, and getting a new ribbon cable through the epoxy?
So what if you can't replace a screen on an epoxy-filled cell phone? That's a small price to pay for knowing that your camera will survive if you take a one-way trip to crush-depth.
Air is the only substance that compresses to any significant amount. When you apply pressure to a battery from one side it may deform, but I would expect that the battery wouldn't be crushed because the pressure is from all sides - unless of course there is air in the battery - I'm not a battery chemist so take everything I say with some salt.
Note that I would expect water to seep into a battery at this pressure and that will cause all kinds of issues - including chemical fires underwater. Just not crushed batteries.
When was the last time your phone stopped working due mechanical PCB damage?
Typically the limiting factor on your phone is the screen breaking, your battery life getting too short, wear and tear on components like buttons or the charging port, and factory defects. Epoxy isn't going to help with any of those. The only thing it would help with is exposure to water, but if other parts of your phone like your screen aren't water proof, what's the point?
Epoxy adds weight and manufacturing cost. It introduces design challenges as you need to balance the thermal expansion of the various parts. It's an extra step that can go wrong, and makes repair of other defects far more difficult. What benefit is there for the typical consumer that outweighs these costs?
To add to that. My son got his phone caught in a reclining chair without realizing it. The fact that the phone bent in half instead of destroying the chair is a nice bonus. Replacing the phone was cheap, replacing a chair would not have been — yes, both are insured, but replacing/repairing a chair takes a hell of a lot longer.
And for just a few bucks per month, it can be insured and replaced for a couple hundred bucks. My chair is also insured through homeowners insurance (the US equivalent name, called something different here in the NL), and they would give me the value of the chair… but now I have to find it again, get it delivered, take my old chair to the dump, etc. The phone was a quick visit to the Apple Store and restore from backup.
Added cost and weight are two things that would put off consumers. The phone would also be neigh irreparable, but consumers don't seem to care for that other than replacing their screen.
I'm not talking about which methods are being used, I'm talking about which methods could be used. Further, potting, where you let the epoxy drip off, gives you a conformal coating.
Conformal coating is much less viscous and would leave a layer orders of magnitude thinner then letting potting epoxy drip off. It's not at all comparable.
Because then it’ll become unrepairable. It’s a tricky trade-off that cell phones/tablets-laptops have to deal with.
Easy to repair vs durable vs cost/time.
And Apple (for example) cares about repairability because they also need/want to repair phones (warranty/trade in for example)
That can be avoided by filling it with a fluid that the repairman can simply drain instead.
People hydromod digital quartz wristwatches by filling them with oil. This gives them truly absurd water resistances and even improves the readability of the screen somehow.
Although the entire enclosure was shaken around enough to tear bits off the PCB via sheer inertia and crack the CPU (hence the need for the recovery process described).
Write an image to a smallish SD card using dd (to remove most of the blocks from wear-levelling circulation), mount without -noatime, and you should be able to get the lifespan down to a few hours!
On iOS, every browser is required by Apple to use WebKit. I just tried it again myself and FireFox on iPhone has no ublock Origin add on possibility.
Firefox Focus does work as an alternative.
Apple created a special system-level API for Safari Content Blockers. Apps like Firefox Focus, AdGuard, 1Blocker, Wipr can register filtering rules with Safari using this API. That’s why Focus can block ads/trackers inside Safari if you enable it under Safari
In EU only, and to my knowledge no mainstream browser vendor has shipped a real browser with their own engine yet. Too much work, and they need to maintain two different versions (EU and non-EU). The whole situation is a mess.
There are countless free and paid options on iOS too
Firefox Focus, Brave
AdGuard Pro, $9.99 once and you can use any blocklist you want (you can just copypaste from uBlock Origin if you wish) and it works system-wide with Safari
Lockdown app for ios - if you arent using your single active vpn connection on iOS lockdown app runs as a local vpn and even app telemetry, ads everything is blocked. Many options on ios, sir.
Figure 3 from the report- that's an Adafruit sensor module on a 3d printed bit of plastic with a teensy-brand microcontroller just sitting in there! Actually, the entire electronics enclosure appears printed.
Very funny to see in what I assume is a million-dollar product.
This is a camera outside the sub, and the electronics are inside sealed enclosure. If that had any leak for water to get in some conformal coating isn't going to save you, it will get pancaked.
Of course random arduino module and teensy used for product is amateur hour, even for low volume production. They must have crazy margins on that camera and producing custom board is very cheap.
>They must have crazy margins on that camera and producing custom board is very cheap.
It's expensive in time and expertise to do a custom board, and to debug a custom board. All to what, save 20$ on a bom which might not even be 1% of the profit per unit?
Far more efficient to just ship the dev board. They could have perhaps picked a better dev setup to start with, but if it looks stupid and it works...
Point isn't to save BOM cost, margins are high already and they can eat into them.
Unless they are making literally less than 10 of those, custom board will be easier to manufacture than that mess of dev boards and more reliable than random wires and headers all over the place. Plus they can spend money where it makes sense, using better regulators etc.
Only if you happen to have the relevant skill set.
Speaking as someone capable of designing the mechanical hardware and who is broadly electrically savvy but who is most definitely not an embedded engineer: I could bang out a few hundred hacked together dev boards in week, but doing a custom board would take me a few months. Starting with reading 'Prototyping PCBs for Dummies'.
Also it's actually simpler than it might seem, especially if you are not doing anything high-power or high-frequency (which, again, if it's a bunch of breakout boards connected over 0.1inch headers they clearly aren't).
Watch a few YT videos, copy-paste the reference schematics for all those boards, delete what you don't use and you are almost done :)
If a hardened camera can survive, I'm surprised subs don't have a floating black box that can survive an implosion and then float to the surface and begin emitting a radio signal.
I guess the trick would be finding a way to securely attach the black box in a way that would ensure its release in a catastrophic disaster.
The ones that aren’t accompanied by a surface ship are military, and they really don’t want anything that might automatically deploy at the wrong moment.
I'm not aware of anything quite like that, but most submarines have something like a Rescue Buoy [0], Submarine Emergency Position-Indicating Radio Beacon (SEPIRB) or Submarine Emergency Communications Transmitter (SECT). I think those might differ based on whether they're attached by a cable and allow communicating to the submarine, or just broadcast a distress signal with the position. In any case, they're designed to be automatically deployed in the event of an emergency or catastrophic event, and based on this Quora answer [1] they're attached by an independent mechanism with a timer which has to be regularly reset to stop it deploying. I think it might be a clockwork mechanism, with an electronic alarm when it's about to go off to remind the crew to wind it.
There are a zillion applications for black-boxes, so why not start somewhere more accessible and with more impact? Your own house, car, and person, for instance. Think of how many elderly people die at home and nobody knows the details leading up to it. I'm being a bit facetious here - perhaps we don't need to know in those cases, nor in the Titan case. It's not as if there could be any data there which advances submarine safety - unless somebody is planning to build a Titan v2 with the same technology, marginally improved safety, and similar lack of testing?
Just fyi - if your car has been manufactured in the last 10 years by a modern corporation(ie - not a Lada) it will have a black box already that records everything about the car's systems. They don't record sound from the cabin, but your speed, throttle position, movement of the steering wheel - it's all recorded.
I mean I have them in both of my cars, but in some places like Austria it's straight up illegal to have them in your car. And in some others like Germany you can have it but the footage is inadmissable in court because the other person never agreed to be recorded. The built in telemetry recording is legal everywhere though.
There were glass spheres outside the main pressure vessel containing electronics and filled with mineral oil that were damaged during the implosion, but the electronics survived mostly intact. This probably would have been a good option for a "black box" recorder. Scott Manley discusses the spheres in his video.
He ridiculed anyone who told him he should, taking it as evidence that he was disrupting an industry.
There is something slightly romantic about dying in such a way that his body turned to mist and floated away in the current. A bit like having your ashes spread at sea. With fewer steps.
if he was alone i would absolutely agree with you full stop.
but it looks like they may have been entirely underselling just how backyard amateur their company was just to get people to give them money.
i totally adore books and documentaries and tales of explorer types pushing their ideas to the limit but it quickly crosses a line when they:
a) downplay dangers to innocent people, and
b) refuse to understand their own ignorance and believe the people who have already literally done their idea are somehow “fools”
the person who is trying to “disrupt” but doesn’t have a deep understanding of the often very good reasons why an industry may do things a certain way is the fool. not the ones who already repeatedly make it work.
we need to encourage adventurousness but also wisdom.
> (Image credit: Scott Manley, David Case at the NTSB)
Why are they crediting Scott Manley instead of just grabbing the images from the reports themselves? It sounds to me as if they wrote an article based on the video, and not based on the report. Very lazy journalism.
Yes because external storage is much larger, and theres nothing more annoying than being in the middle of doing some science with 30 other bits of complex equipment, and then the camera stops working with storage full errors and youre 7000m underwater in a cramped sub trying to navigate a camera UI to find the setting.
Configure your systems so they are in the configuration that is less likely to cause random disruption in the field.
Which makes me wonder why they bother with the SD card at all. What was it meant to be storing? If it is not intended to be the real storage area, why not just have it in a loop, constantly over-writing the oldest material?
The camera was an off the shelf part (from a very specialty shelf I suppose). It had an SD card built in because some people might not have a thing to stream to; it's probably good for demos and cheap enough to be good for a bullet point. Given the rest of the components inside they probably had enough margin that they weren't optimizing for costs. The value add was in the pressure vessel, and that seems to have mostly worked.
> Somewhat disappointingly, the images and videos shared in the report were taken in the vicinity of the ROV shop at the Marine Institute, also in Newfoundland. The location was the logistical base for Titanic dive missions. No deep-sea shenanigans around the Titanic wreck were revealed.
Wouldn’t it have been streaming it to disk without creating the file? Kinda like how if your camera dies while it’s recording, there is no recording. You have to manually recreate the file.
I know the default policy is to use the source headline, but I feel the mention of $62 is kindof unfair. It evokes cheapness. Many many aspects of the craft were cheap and underbuilt, but it’s not clear to me that the memory card was one of those areas. Seems like it was an actually useful COTS part
Isnt the weakness here that there was nothing encrypting the actual key? On a laptop luks key stored in a tpm would usually be encrypted using your passphrase
The NTSB report noted that if the TrustZone secure enclave system was being used, then yeah this data would be toast.
But it speaks more to Oceangstrs negligence that this situation even existed: why wasn't any potential encryption keys escrowed ashore to ensure they could be recovered later? This shouldn't have even been an issue.
It seems the manufacturer of the camera didn't even know (at least in the part of the org communicating with the NTSB) that their storage was encrypted. In any case the media recovered were from testing/non-dive environments, and during an actual dive footage would presumably be recorded directly to the onboard computers (which were irrecoverably destroyed).
Oceangate should take the blame for a lot of things but probably not this.
Them sandisks need to be pretty tough to not vaporize whilst cooking your USB ports. Ever notice how infernally hot they get under large file transfers?
I see a lot of discussion in this thread stemming from some confusion+not reading the actual report[0].
Some key points:
1. The Camera+Card was encased in a separate enclosure made of titanium+sapphire, and did not seem to be exposed to extreme pressures.
2. The encryption was done via a variant of LUKS/dm-crypt, with the key stored on the NVRAM of a chip (Edited; not in TrustZone).
3. The recovery was done by transplanting the original chip onto a new working board. No manufacturer backdoors or other hidden mechanisms were used.
4. Interestingly, the camera vendor didn't seem to realize there was any encryption at all.
[0] https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...
Unless I misread the article, the key was stored in the NVRAM and not the TrustZone.
IIRC, the article stated that if the key(s) had been stored in the TrustZone then the data would have been irrecoverable.
Good catch; it was somewhat ambiguous in the report.
[dead]
> 1. The Camera+Card was encased in a separate enclosure made of titanium+sapphire, and did not seem to be exposed to extreme pressures.
I wonder what the price of the enclosure was then. Feels a bit like click bait...
It might have been filled with mineral oil, those external enclosures often setup that way so that the enclosure is less extreme to manufacture. Not sure if that would work for camera lenses though unless those were also filled.
As someone who was on the product team for a 6000msw video camera, it's probably not filled with mineral oil. I doubt anyone makes subsea camera bodies/optics completely from scratch, and off-the-shelf units are not designed for pressure. In ours we used Sony camera internals, the enclosure was atmospheric, filled with dry nitrogen to reduce condensation, and the sapphire lens was designed to resist 12500psi and reduce distortion from the air-->sapphire-->seawater interface.
Hmm ok that would have explained why the SD card wasn't damaged if it were because when the vessel is filled with mineral oil it wouldn't have imploded like the main body of the craft.
Alot. Just google cameras for deep sea and space. Several companies make these and despite all the covering up, none of the tech is that special.
Clickbait? Where? How? It's literally in the NTSB report, and it's not like, a crazy concept?
Here; the title focusing on the price is implying that the cheap SD card survived ocean floor environment alone. A surprising amount of stress for its price.
Instead, a pressure-proof deep sea camera module was found at the wreckage site. It’s less interesting that an expensive thing rated for ocean depths was intact at ocean depths.
Its like “missing child found after 4 days in Alaska temperatures!”
gasp! How did they survive!
“The child was on holiday in their grandparents’ holiday log cabin, with their grandparents, a log fire, food, water..”
Oh. Clickbait. Hiding the boring bit to make the story appear more of a tease.
I noticed a pattern a few months ago in my phone's newsbait feed of headlines in the format "[Large familiar company] to close 500 stores on [date]" — and then below the fold "because it's [Independence Day, Rosh Hashanah, etc]" or "because they are moving to summer hours" or whatever.
> Here; the title focusing on the price is implying that the cheap SD card survived ocean floor environment alone. A surprising amount of stress for its price.
I certainly did not read that implication into the title, so it's entirely possible that the author didn't mean it.
That was a laugh I needed and I’m using that in the future. The timing was perfect and I lost some coffee. Thank you!
The title of TFA is clickbait - it tries to make it sound like the memory card itself withstood extreme pressures.
If the encryption was that easy to bypass, was it worth it at all?
The manufacturer didn’t even know encryption was enabled, because as long as the camera was working, it would just provide all files over USB without any encryption.
It was basically enabled by accident, and the only thing it prevented was recovery of files directly from the SD card when the camera was damaged.
There are some reasons you'd want to encrypt even without a secret key. One is it makes it easier to erase data (just erase the key).
It also makes bit flip errors a lot more obvious, which is another way of saying harder to ignore, so that can go either way.
Can't bit flip errors also destroy encrypted volumes much more easily?
I think it depends. Encrypted filesystems typically encrypt contents of each file separately - that way you don't need to read / write the whole disk to read it write any individual file contents. Of course that means metadata may be in plain text or may be separately encrypted - again possibly folder by folder instead of all metadata at once. Exact details would vary with different file system encryption schemes.
Whereas if you image the disk and encrypt the image properly, that gives you all the great confidentially guarantees but no random access.
> Encrypted filesystems typically encrypt contents of each file separately - that way you don't need to read / write the whole disk to read it write any individual file contents.
Ah, that's not true of "full disk encryption". It usually encrypts the disk blocks.
File-based encryption is stronger; you can use different protection classes on different files, you can use authenticated encryption, etc. iOS does it this way and I assume other systems have caught up, but don't know any in particular.
File-based encryption leaks metadata (which in some cases is bad enough to render it unusable).
No one said you had to leave the FS itself unencrypted.
Most FDE systems are not authenticated so you would only lose one block (16 bytes for AES). Can this be bad? Yeah, but it's not that bad for data recovery.
Not to mention that most drives start having issues with dead sectors rather than bitflips, and that's (usually) 4K.
Encryption does not make bit flips obivous, authenticated encryption would.
A single bit flip would mess up the block, and hopefully the rest of the stream and the padding, no?
Most unauthenticated encryption modes only mess up a few bits of a block, sometimes the following block too. A few only flip the exact bit in the plaintext.
Sure. If the card was recovered without the camera motherboard then the decryption key would not have been recovered.
Stealing a camera is much harder than stealing an SD card out of a camera.
Citation needed. It might be slightly easier, but most cases where you can get part of the camera, you can get the whole camera. This isn't a little point-and-click with a handy spring-loaded slot either.
Yeah but the Camera's owner is much more likely to notice "my camera is missing" than "the SD card is blank for some reason... the SD card must have failed"
EDIT: The linked PDF has a photo, the camera literally opens up to access the SD card.
The camera's (former) owner may very well notice, but that will have little effect. It's much more common that cameras (security, photography, phones) get stolen with cards inside, rather than someone extracting the card and leaving the camera.
This is professional equipment, used for surveys. Think espionage, not consumer hardware.
Worth mentioning that I would immediately know if a different SD card was in my camera the moment I turned it on or ejected the card. If somebody knew to buy the same exact model and storage size that would be truly impressive.
Industrial espionage is far far more often done by hard work then being clever. Checking the SD cards you use and buying matching ones before executing a swap isn't noteworthy.
If you look through my case of SD cards I have all sorts of sizes and makes/models. I also have a procedure for dumping and formatting, and someone who is handling this most likely would as well. The moment the storage on screen looks funny or I see the card I don’t recognize, I’ll notice.
I’m not saying it’s impossible or I’m somehow immune to being tricked, but you would be surprised how easy it is to leave evidence of tampering through the simple act of trying to take an SD card, whether you swap it out or not. And people like me who handle them regularly would likely notice it in a split second. Maybe we won’t even catch the perpetrator, but it would not be written off as “oh interesting I guess it didn’t record,” if for no other reason then I would go straight into CYA mode and start looking for any hint that I didn’t make a mistake.
At that point if you have a basic security SoP in place and adhere to it you can start auditing.
Anywho again nothing is airtight and a determined individual could of course get past me. But you would be surprised how many hurdles they have to get over, and it certainly wouldn’t go unnoticed and be brushed off as a read/write quirk, that’s really all I’m saying
0. They were too cheap to use an industrial grade SD. Mind boggling.
If you read the article, the SD card was placed there by the camera manufacturer and then the device was sealed so it would withstand pressure, and then sold to divers. Blame the camera manufacturer's engineers.
Seems like the SD card of all things performed just fine, so it hardly seems like the weak point.
They are surprisingly resilient! There was a blog post a couple of years ago by someone who found a digital camera in the sea which must have been in the water for months. The author could look at the photos to see if they can find the original owner of the camera.
> No deep-sea shenanigans around the Titanic wreck were revealed. Manley explains in his Twitter thread that “the camera had been configured to dump data onto an external storage device, so nothing was found from the accident dive.” Nothing particularly pertinent to the tragic accident, that is.
This is about camera hardware and how it survived. It provides no information or footage about the incident (in case you were looking for it like I was).
Thank you, this post saved me time.
The NTSB's original report has more detail on how the SD Card was encrypted and how the NTSB managed to decrypt it:
https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...
The System on Module board is an Inforce 6601 SOM. [0]
It uses a Qualcomm Snapdragon 820 and they provide prebuilt Ubuntu Linaro distros for it, preconfigured for the board.
The camera manufacturer likely just tossed it straight in as configured and thus didn't know how the full disk encryption was setup.
This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'.
See also: almost any specialty medical electronic device ever manufactured.
[0] https://linuxgizmos.com/tiny-rugged-com-runs-linux-or-androi...
> This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'.
> See also: almost any specialty medical electronic device ever manufactured.
These are not design mistakes.
When building products in short runs and where the costs of part have little impact on your margins compared to R&D, it completely makes sense to go for a full computer rather than bother with embedded development where everything is more complicated. Medical also has to deal with certification which is a much more significant concern than saving on parts and will often reuse already certified components.
I'll admit I only watched a video on it not the report, but it had pictures reportedly redacted at manufacturer request. It showed a teensy 3 and some adafruit qwiic board in there. Obviously the real engineering is in the enclosure. Otherwise it could just be a webcam. But still, it's clearly not a very in depth electrical design. I'm all for SoMs if you can but they don't guarantee you the adventure of custom hardware bringing moving through all the software stacks and whatnot.
No serious commercial product should be using a Teensy under basically any circumstance.
Can I ask why? I'm not really into microprocessors.
Usually (not always), something like a Teensy or a Pi Pico or an Arduino is treated like a development board for prototyping.
A person builds out their circuit using hardware they can solder/wire-up by hand on a workbench, maybe even with relatively-giant solderless breadboards, to prove the concept and the general design.
And a dev board can be great for spinning a few prototypes. It's quick to get started (code can begin being tested on-chip after just plugging in a USB cable), and to try different things and to make (and correct!) mistakes. (Blow up a Teensy? No worries; just grab another from the drawer, try not to make that same mistake again, and keep moving -- no esoteric soldering required)
But when the design is finished-enough and it becomes time to spin up custom-built PCBs for a final product that will be sold, a separate dev board like a Teensy tends to lose much of its initial charm.
Instead, it's more-typical just put the microcontroller IC plus whatever supporting hardware is necessary for the overall device's actual functions on the main board. Don't need USB, or an Ethernet PHY, an LED, a button, or a separate voltage regulator? Want more or less flash? When including the MCU on a board of one's own design instead of a kitchen-sink dev board, one is empowered to use exactly the parts that are required.
This can save a substantial amount of space and greatly improve the flexibility of the layout, while also improving mechanical and electrical robustness by having fewer connections between the MCU and the world around it. Plus, fewer parts tend to be less costly than more parts are.
(But again, it's not always done this way. This camera from the submarine is an example of one instance where the whole dev board was put inside of a finished product. Sometimes that's a good idea, and sometimes it isn't. I'm not attempting to suggest that it was or was not a good move in this instance.)
That’s entirely true for manufacturing at scale. I really doubt they are selling a lot of these.
I suspect you're right about the quantities. In support of that notion, when looking closer at the (linked in another user's comment) PDF of the report, I can see that a lot of this camera's internal structure quite clearly appears to be the product of an FDM 3D printer. This suggests that quantities are low.
And I don't know when that camera was manufactured or designed.
But these days, it's possible to get even hobbyist-quantities of custom PCBs delivered with difficult-to-solder ICs installed from sources like JLCPCB.
(Depending on the features and functions wanted, it doesn't take a whole lot of extra parts to get an MCU to do its thing: There's not a ton of parts on a Teensy to begin with.)
JLCPCB will stick an IMXRT1064 and an oscillator on 5 custom PCBs for you for <$100 - the Teensy is basically $25-$35 for the same thing.
$25-35/ea, so well over $100 for 5 boards.
Everything you said makes sense except you haven't explained why you can't just seal up a Teensy in an enclosure and sell it that way, except for "you're not supposed to do that". Are Teensies prone to random failure or something? Because if they just work and you're only selling <50 devices for extremely specialized nieche then I really don't see a problem with this?
(I previously wrote something rather crude. I am attempting to revise that into something more positive.)
In this context, all people are free to do whatever they want. It is beyond me to suggest that any person cannot do a thing.
Well, but it's not about whether someone can or cannot do something. Since you seem experienced with these devices I'm just asking if there's any technical reason why this might be a bad idea other than the fact that it just doesn't seem like a very professional thing to do. Like for example even though I'm far from an expert I know Raspberry Pis would be awful for any commercial application because they are notorious for killing their SD cards rendering the device useless.
The Teensy wasn’t engineered, tested, rated, or certified for any sort of continuous duty, let alone within a pressurized O2-enriched environment (assuming it was inside the vessel), especially not within deep sea Helium-enriched environments (that have been shown to break things like MEMS devices), and present unnecessary risks for an entirely inefficient choice (see: comment above, Teensy’s are ~$30-40/ea where small PCBs populated with the same circuit features can be had for under $20).
I’m probably not as qualified as the person you replied to, but that’s my intuition as someone with a passing familiarity with electronics engineering (I have an associates degree in EE).
People do whatever they want. It doesn't have to make sense.
Perhaps disturbingly: I even know of one bit of critical public safety communications infrastructure that is is expensive, low production volume, and has a Raspberry Pi 3 embedded inside. I won't name names because that's getting a little too close to home for my liking, but I was quite surprised to find this inside of a very nice waterproof box with chonky, expensive, olive-colored milspec connectors to connect it up to the outside world.
Which, well: Yeah. There's a ton of good reasons not to do that. But building a whole Linux system on a custom board using individual parts is hard, so it can make sense to buy someone else's work instead.
Except... that's what the CM3 is designed to provide, including on-board eMMC instead of an SD card. I'd not have been surprised at all if there was a CM3 in there, but there is instead an entire Pi 3.
But MCUs, like on the Teensy, aren't like that. They aren't hard to integrate on a custom board like the Broadcom SoC on a Pi 3 or CM3 is.
The primary purpose of an MCU is not to be stuffed onto a dev board like a Teensy, but instead to be stuffed onto the board inside of a microwave oven or an air fryer or a fancy remote control and be easy to interface with other things and to program.
It really doesn't take much to get them going: Some require external ROM or flash, but a lot of them have internal flash memory and only need power and programming pins wired up to let them run code and do whatever IO is needed within a system.
This camera already had at least one very custom board inside. It could have integrated the MCU, as well, instead of the kitchen-sink Teensy.
Doing so is not just style points; it's quite often easier, cheaper, and more flexible.
This allows a person to use all of the IO pins on the MCU to do stuff with, instead of just the functions that the designer of a dev kit decided to build out through whatever interfaces they decided to include.
[dead]
The 3D-printed (and hot glued?) part in Figure 3 further support this theory (not that 3D prints can't be used in production).
Indeed this is massively overcomplicated, as one only needs to see what dashcams use to know that you don't need, or perhaps want, an entire OS on it.
Does not leave SubC in a particularly flattering light...
They had no idea how their own product worked. They didn’t even know it used encrypted storage.
This was either outsourced or done by some junior engineer who was putting pieces together like it was another Raspberry Pi project that just needed to kind of work.
The longer I last in this world the more products I realize are the result of telling a few people who don't know what they are doing to "make it kind of work."
That’s my entire experience in embedded. Everything I get from other companies basically looks like an internship project right down to the pointer arguments with unspecified bounds on the function calls. One of the companies we bought hardware from keeps representing things are working when they only work on devices in the lab. Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
> Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
While I agree with your sentiment that there's a lot of poor software engineering in embedded space (especially in consumer-oriented novelty products, less so in established fields like industrial or telco), I can't but wonder what's wrong with Yocto? In my experience, it's quite the opposite: Yocto is the quickest path to get the firmware for a new device assembled, once you have climbed its pretty steep learning curve. I have built a few homebrew firmware build systems out of Debian and make/shell scripts (not my choice), you pretty quickly find yourself reinventing half of the stuff that Yocto does out of the box, but it's all bespoke, janky and hard to maintain. While with Yocto you just take the vendor's meta layer for BSP, put your application in another, and it bakes you a set of flashable images on the other end, complete with SDKs and other goodies for your dev workflow, reproducibly. It doesn't get significantly more sophisticated once you start to need kernel customizations, firmware updates with A/B partition layout, readonly rootfs, manage board- or customer-specific variants and other features that are very common in embedded systems but poorly or not at all supported in standard distros.
The problem isn’t OE per se but every vendor pulls in their own janky patches for everything and you find yourself fighting the build system to cut packages and reduce the filesystem size. It’s also really slow and it’s frankly massive, requiring tens or hundreds of gigabytes of stuff to yield less than a gigabyte of system image. And at the end of the day if a vendor breaks something from another vendor you will find yourself struggling to figure out why. Older versions of their documentation make statements that small teams should avoid Yocto for these reasons.
If you don’t already know anything about how to do these things in Linux Yocto is good for it. But if you try to swim against the current of the way the layer authors want you to do any of those things you will find it very challenging.
Take read-only root, for example. Usually it’s a one-line change in a config file plus an additional mount should you want an overlay in volatile memory. I don’t see how pulling in layers and config files does anything other than obscure how that works.
For a good example of what I hate about it, try building from Meta-Intel without graphics drivers.
What kind of hardware projects do you work on?
I'm mostly in the software space but in the past few years I've been doing a lot more embedded stuff, and the trend I notice is that companies are making great hardware, and then completely ruining its usefulness with bad software and firmware. It's kind of mind blowing to me because I always considered software to be the easy part of making a product, compared to, you know, etching microscopic patterns onto sand to make magical transistors appear in just the right way to do the task you want.
It's all about what works enough according to a rather low bar.
It's almost as if hardware was developed by actual engineers, unlike, you know, the other half of the pie.
This is in fact the case.
It survived the pressure, does the rest matter?
In this context? Certainly. This is more examples of their using untested, uncertified equipment. Anyone who’s ever worked in O2-enriched pressurized environments should at least be instantly concerned about whether this device had been properly rated for fire prevention.
Scott Manley’s 45 minute video covering the NTSB report this information comes from went up today, it’s quite interesting.
https://youtu.be/qMUjCZ7MMWQ
It continues to amaze me how indestructible SDCards are.
It's a solid piece of silicon encased in epoxy, so there's nothing really to get crushed. Contrast this to something like a cellphone that's made of hundreds of separate parts and has void space that will get crushed.
Are consumer grade cards really reliable though? Not so much against physical damage, but of data integrity over extended periods? "Industrial" SD cards can be 10 times or 100 times more expensive than consumer grade cards.
So were the flash chips on the SSDs they found. It didn’t save them.
the plural part is the key here, the bigger area means any uneven forces have more chance of bending and cracking stuff.
It was also in casing already prepared for that type of environment
This comment made me wonder how much easier proximity fuzes would have been to develop in WW2 had they had transistors (or integrated circuits). I assume making modern solid state electronics 20,000g shock resistant is much easier than doing the same to vacuum tubes.
No need to wonder, proximity fuzes are still used today. And yes, they are much smaller, cheaper, more reliable, and precise.
I was wondering how much easier it would have been, specifically in dollar terms. Order of magnitude?
Why isn't a cellphone filled with epoxy?
How would you do screen replacement? That is a common repair since people drop their phones and currently you can get your phone repaired by some teenager in a booth at the mall. If you fill the phone with epoxy, how are you detaching the screen, and getting a new ribbon cable through the epoxy?
Just like they do it today - a lot of grinding, swearing and overall understanding what the civilization is going in not quite the right direction.
So what if you can't replace a screen on an epoxy-filled cell phone? That's a small price to pay for knowing that your camera will survive if you take a one-way trip to crush-depth.
Is this sarcasm? 99.999% people will never take it beyond a meter in a pool.
Poe's Law never fails. I should have accounted for that.
use pogo pins or a board to board connector
Which means air space that can get crushed. Either the phone is solid or it isn't.
> Which means air space that can get crushed
Would note that air isn't the only substance in a phone that compresses under 38 MPa. (Batteries come to mind.)
Air is the only substance that compresses to any significant amount. When you apply pressure to a battery from one side it may deform, but I would expect that the battery wouldn't be crushed because the pressure is from all sides - unless of course there is air in the battery - I'm not a battery chemist so take everything I say with some salt.
Note that I would expect water to seep into a battery at this pressure and that will cause all kinds of issues - including chemical fires underwater. Just not crushed batteries.
I'm sure there are some companies who want to do that, as long as they can convince people it's better for security or something.
When was the last time your phone stopped working due mechanical PCB damage?
Typically the limiting factor on your phone is the screen breaking, your battery life getting too short, wear and tear on components like buttons or the charging port, and factory defects. Epoxy isn't going to help with any of those. The only thing it would help with is exposure to water, but if other parts of your phone like your screen aren't water proof, what's the point?
Epoxy adds weight and manufacturing cost. It introduces design challenges as you need to balance the thermal expansion of the various parts. It's an extra step that can go wrong, and makes repair of other defects far more difficult. What benefit is there for the typical consumer that outweighs these costs?
To add to that. My son got his phone caught in a reclining chair without realizing it. The fact that the phone bent in half instead of destroying the chair is a nice bonus. Replacing the phone was cheap, replacing a chair would not have been — yes, both are insured, but replacing/repairing a chair takes a hell of a lot longer.
I think most would disagree XD.
Phones these days are often more expensive than the chair and can be pretty inconvenient to replace, especially if you have nonrecent backups.
Yeah not sure about you guys but me and everyone I know buys their stuff in ikea where a chair definitely doesn't cost more than a good cell phone
I got an Eames chair recently and would be devastated if my phone damaged it!
And for just a few bucks per month, it can be insured and replaced for a couple hundred bucks. My chair is also insured through homeowners insurance (the US equivalent name, called something different here in the NL), and they would give me the value of the chair… but now I have to find it again, get it delivered, take my old chair to the dump, etc. The phone was a quick visit to the Apple Store and restore from backup.
The GoPro Session actually took this tack to achieve waterproofness without a secondary case.
The heavy components on a cell phone PCB are reinforced with spot applications of adhesive to the PCB.
Filling the entire cell phone with epoxy wouldn’t help. The parts that break on drops are external like the screen.
This SD card was enclosed in a sealed metal container so it wasn’t exposed to water.
OTOH, adding epoxy on top of everything else would probably only reduce their iFixit repairability score from 1 to 0, so...
A conformal coating wouldn't give much more weight.
A conformal coating isn't "filled with epoxy", which is the concern I was answering.
There is very little empty space in a phone, so conformal coating is practically the same as filling it.
Anyway, I wasn't disagreeing, just reasoning a bit further.
The point of filling it is to remove the compressible empty space so that large pressure gradients won’t crush it.
No, conformal coating and potting are extremely different things done for different reasons.
I'm not talking about which methods are being used, I'm talking about which methods could be used. Further, potting, where you let the epoxy drip off, gives you a conformal coating.
Conformal coating is much less viscous and would leave a layer orders of magnitude thinner then letting potting epoxy drip off. It's not at all comparable.
You are talking about a conventional conformal coating.
Neigh?
I didn't notice that, I was dictating to Gboard. If that's what was heard, then I should probably go eat some hay and get my tail brushed.
I think they meant “nigh on irreparable“.
Some claim we are centaurs, we say Neigh!
This joke seems like it would make more sense if centaurs didn't have human vocal tracts.
Because then it’ll become unrepairable. It’s a tricky trade-off that cell phones/tablets-laptops have to deal with. Easy to repair vs durable vs cost/time. And Apple (for example) cares about repairability because they also need/want to repair phones (warranty/trade in for example)
Filling dive watches with oil (hydro modding) is pretty popular. It mainly helps with visibility but also increases the depth rating.
Well, most cellphones aren't subjected to the conditions found under three miles of frigid sea water. Epoxy is also really, really expensive.
It's just not necessary, while having reliability problems of its own.
Because then it gets a 0/10 repairability score on ifixit :)
That can be avoided by filling it with a fluid that the repairman can simply drain instead.
People hydromod digital quartz wristwatches by filling them with oil. This gives them truly absurd water resistances and even improves the readability of the screen somehow.
“My phone needs an oil change”
“I dropped my phone and all the oil leaked out making a mess”
Yeah that’s not going to happen my dude.
Thermal concerns perhaps - how does epoxy dissipate heat?
Some types of epoxy actually conduct heat quite well.
I don't need any extra grams in my phone!
Shhhh - don't give Apple ideas.
That would be a problem for the mic and speaker, and has relatively few use cases.
Imagine how much drama they could have avoided if they filled the entire submersible.
Say for argument’s sake there was a small air bubble in the resin. Couldn’t that result in cavitation?
So that's the next phase of making devices thinner? /s
It wasn't in the crushed part, it was in the camera's shell, and the camera was mounted outside, if I understood properly.
And:
> This still and video camera is rated to withstand depths up to 6,000m (19,685 feet, 3,281 fathoms)
Unlike the Titan sub...
Why didn't they just build the sub out of camera enclosures? /s
The picture looks like the camera + storage SD card were in a sealed metal tube that was untouched.
It clearly received a nasty shock when the sub imploded; that's why the internal components were so broken.
Although the entire enclosure was shaken around enough to tear bits off the PCB via sheer inertia and crack the CPU (hence the need for the recovery process described).
The SDCard that was in another sub, properly constructed from titanium not carbon. The sub housed a camera, no humans.
Heat and wear are the greatest dangers to flash memory, and this was found in a cold dark place, with presumably plenty of life remaining.
It continues to amaze me how indestructible SDCards are.
Until they're sold as supplemental hard drives (cough Transcend Jetdrive cough). Then they'll fail if you even look at them strangely.
Put one in a Raspberry pi and it will be dead in a month.
A month? Those are rookie numbers!
Write an image to a smallish SD card using dd (to remove most of the blocks from wear-levelling circulation), mount without -noatime, and you should be able to get the lifespan down to a few hours!
remember the noatime mount option for the root fs!
It also amazes me how incredibly unbrowseable tomshardware is now with all the ads and pop-ups.
It also amazes me that people are using the internet w/o an adblocker in the year 2025
I haven’t bothered working out how to install one on mobile. I just don’t visit websites with shitty ads.
Just use the Brave browser. No plugins necessary.
It has poor compatibility on the iOS version that I've got installed, sadly.
Firefox on mobile supports uBlock Origin
On iOS, every browser is required by Apple to use WebKit. I just tried it again myself and FireFox on iPhone has no ublock Origin add on possibility.
Firefox Focus does work as an alternative.
Apple created a special system-level API for Safari Content Blockers. Apps like Firefox Focus, AdGuard, 1Blocker, Wipr can register filtering rules with Safari using this API. That’s why Focus can block ads/trackers inside Safari if you enable it under Safari
use orion browser? or ubo lite on safari
> On iOS, every browser is required by Apple to use WebKit
Didn’t this change recently?
In EU only, and to my knowledge no mainstream browser vendor has shipped a real browser with their own engine yet. Too much work, and they need to maintain two different versions (EU and non-EU). The whole situation is a mess.
Only for EU.
Safari on iOS support uBlock origin too
I think you mean HN readers.
i was also in shock, then someone reminded me there are iphone users.
the horror.
paying thousands of dollars just to be forbidden to block ads.
?
There are countless free and paid options on iOS too
Firefox Focus, Brave
AdGuard Pro, $9.99 once and you can use any blocklist you want (you can just copypaste from uBlock Origin if you wish) and it works system-wide with Safari
etc
Lockdown app for ios - if you arent using your single active vpn connection on iOS lockdown app runs as a local vpn and even app telemetry, ads everything is blocked. Many options on ios, sir.
what? there are many fantastic ad blockers on ios. Weird thing to crow about.
With UblockOrigin blocking the ads, there were no ads and pop-ups.
Figure 3 from the report- that's an Adafruit sensor module on a 3d printed bit of plastic with a teensy-brand microcontroller just sitting in there! Actually, the entire electronics enclosure appears printed.
Very funny to see in what I assume is a million-dollar product.
On-brand, though. And speak some respect to Adafruit's name! Lady Ada's product isn't what failed.
What was the water condensation situation like in this submarine? Semi bare electronics sounds very very bad.
This is a camera outside the sub, and the electronics are inside sealed enclosure. If that had any leak for water to get in some conformal coating isn't going to save you, it will get pancaked.
Of course random arduino module and teensy used for product is amateur hour, even for low volume production. They must have crazy margins on that camera and producing custom board is very cheap.
>They must have crazy margins on that camera and producing custom board is very cheap.
It's expensive in time and expertise to do a custom board, and to debug a custom board. All to what, save 20$ on a bom which might not even be 1% of the profit per unit?
Far more efficient to just ship the dev board. They could have perhaps picked a better dev setup to start with, but if it looks stupid and it works...
Point isn't to save BOM cost, margins are high already and they can eat into them.
Unless they are making literally less than 10 of those, custom board will be easier to manufacture than that mess of dev boards and more reliable than random wires and headers all over the place. Plus they can spend money where it makes sense, using better regulators etc.
Only if you happen to have the relevant skill set.
Speaking as someone capable of designing the mechanical hardware and who is broadly electrically savvy but who is most definitely not an embedded engineer: I could bang out a few hundred hacked together dev boards in week, but doing a custom board would take me a few months. Starting with reading 'Prototyping PCBs for Dummies'.
or just hire a guy that knows how to do their job.
Also it's actually simpler than it might seem, especially if you are not doing anything high-power or high-frequency (which, again, if it's a bunch of breakout boards connected over 0.1inch headers they clearly aren't).
Watch a few YT videos, copy-paste the reference schematics for all those boards, delete what you don't use and you are almost done :)
Any money spent coating those semi-bare electronics would have been wasted. There's an engineering lesson to be taught here, I'm sure.
Lol, and the censored controller is a teensy 3.2
If a hardened camera can survive, I'm surprised subs don't have a floating black box that can survive an implosion and then float to the surface and begin emitting a radio signal.
I guess the trick would be finding a way to securely attach the black box in a way that would ensure its release in a catastrophic disaster.
The ones that aren’t accompanied by a surface ship are military, and they really don’t want anything that might automatically deploy at the wrong moment.
This was part of what complicated the response to the Kursk disaster - they had a rescue buoy but it was welded in place so it couldn't deploy.
I'm not aware of anything quite like that, but most submarines have something like a Rescue Buoy [0], Submarine Emergency Position-Indicating Radio Beacon (SEPIRB) or Submarine Emergency Communications Transmitter (SECT). I think those might differ based on whether they're attached by a cable and allow communicating to the submarine, or just broadcast a distress signal with the position. In any case, they're designed to be automatically deployed in the event of an emergency or catastrophic event, and based on this Quora answer [1] they're attached by an independent mechanism with a timer which has to be regularly reset to stop it deploying. I think it might be a clockwork mechanism, with an electronic alarm when it's about to go off to remind the crew to wind it.
[0]: https://en.wikipedia.org/wiki/Rescue_buoy_(submarine)
[1]: https://www.quora.com/Don%E2%80%99t-submarines-have-communic...
There are a zillion applications for black-boxes, so why not start somewhere more accessible and with more impact? Your own house, car, and person, for instance. Think of how many elderly people die at home and nobody knows the details leading up to it. I'm being a bit facetious here - perhaps we don't need to know in those cases, nor in the Titan case. It's not as if there could be any data there which advances submarine safety - unless somebody is planning to build a Titan v2 with the same technology, marginally improved safety, and similar lack of testing?
Just fyi - if your car has been manufactured in the last 10 years by a modern corporation(ie - not a Lada) it will have a black box already that records everything about the car's systems. They don't record sound from the cabin, but your speed, throttle position, movement of the steering wheel - it's all recorded.
In my country, people call dashcams "black boxes". They can be quite useful if you also want audio and video.
I mean I have them in both of my cars, but in some places like Austria it's straight up illegal to have them in your car. And in some others like Germany you can have it but the footage is inadmissable in court because the other person never agreed to be recorded. The built in telemetry recording is legal everywhere though.
Probably commerical subs aren't a common thing with lots of regulation.
Just guessing here? :)
There were glass spheres outside the main pressure vessel containing electronics and filled with mineral oil that were damaged during the implosion, but the electronics survived mostly intact. This probably would have been a good option for a "black box" recorder. Scott Manley discusses the spheres in his video.
I guess he wasn’t planning for a catastrophic failure ?
He ridiculed anyone who told him he should, taking it as evidence that he was disrupting an industry.
There is something slightly romantic about dying in such a way that his body turned to mist and floated away in the current. A bit like having your ashes spread at sea. With fewer steps.
I keep thinking about the teenager.
Who didn't want to go and didn't feel safe, but who was pushed to come along by his father because it was his father's birthday.
Super tragic that one.
> There is something slightly romantic…
if he was alone i would absolutely agree with you full stop.
but it looks like they may have been entirely underselling just how backyard amateur their company was just to get people to give them money.
i totally adore books and documentaries and tales of explorer types pushing their ideas to the limit but it quickly crosses a line when they:
a) downplay dangers to innocent people, and
b) refuse to understand their own ignorance and believe the people who have already literally done their idea are somehow “fools”
the person who is trying to “disrupt” but doesn’t have a deep understanding of the often very good reasons why an industry may do things a certain way is the fool. not the ones who already repeatedly make it work.
we need to encourage adventurousness but also wisdom.
> (Image credit: Scott Manley, David Case at the NTSB)
Why are they crediting Scott Manley instead of just grabbing the images from the reports themselves? It sounds to me as if they wrote an article based on the video, and not based on the report. Very lazy journalism.
is this a common setup to have the camera store to external storage device without storing to the SD card as well?
Yes because external storage is much larger, and theres nothing more annoying than being in the middle of doing some science with 30 other bits of complex equipment, and then the camera stops working with storage full errors and youre 7000m underwater in a cramped sub trying to navigate a camera UI to find the setting.
Configure your systems so they are in the configuration that is less likely to cause random disruption in the field.
Which makes me wonder why they bother with the SD card at all. What was it meant to be storing? If it is not intended to be the real storage area, why not just have it in a loop, constantly over-writing the oldest material?
The camera was an off the shelf part (from a very specialty shelf I suppose). It had an SD card built in because some people might not have a thing to stream to; it's probably good for demos and cheap enough to be good for a bullet point. Given the rest of the components inside they probably had enough margin that they weren't optimizing for costs. The value add was in the pressure vessel, and that seems to have mostly worked.
They probably used it for testing only, hence why it had irrelevant footage.
They might have forgot to remove or just didn't care.
Since not everyone reads articles:
> Somewhat disappointingly, the images and videos shared in the report were taken in the vicinity of the ROV shop at the Marine Institute, also in Newfoundland. The location was the logistical base for Titanic dive missions. No deep-sea shenanigans around the Titanic wreck were revealed.
Wouldn’t it have been streaming it to disk without creating the file? Kinda like how if your camera dies while it’s recording, there is no recording. You have to manually recreate the file.
I think that’s what they think. It was being recorded by the onboard PC to its SSDs, which were completely destroyed in the implosion.
I know the default policy is to use the source headline, but I feel the mention of $62 is kindof unfair. It evokes cheapness. Many many aspects of the craft were cheap and underbuilt, but it’s not clear to me that the memory card was one of those areas. Seems like it was an actually useful COTS part
Yeah, and the title doesn't mention that is for a camera either.
This seems perfectly reasonable for camera data storage, and apparently a camera that's manually operated as well.
It'd be worrying if it was storing the binaries to operate the sub, but this is a decent quality SD card for camera storage.
also basically if enough companies agrees on helping the cause your crypto secrets are quite more likely to be exposed...
Isnt the weakness here that there was nothing encrypting the actual key? On a laptop luks key stored in a tpm would usually be encrypted using your passphrase
The NTSB report noted that if the TrustZone secure enclave system was being used, then yeah this data would be toast.
But it speaks more to Oceangstrs negligence that this situation even existed: why wasn't any potential encryption keys escrowed ashore to ensure they could be recovered later? This shouldn't have even been an issue.
It seems the manufacturer of the camera didn't even know (at least in the part of the org communicating with the NTSB) that their storage was encrypted. In any case the media recovered were from testing/non-dive environments, and during an actual dive footage would presumably be recorded directly to the onboard computers (which were irrecoverably destroyed).
Oceangate should take the blame for a lot of things but probably not this.
Dear Tom's Hardware,
Tech users get upset when you make me do what you want instead of what the user wants.
Sincerely,
The Back Button
Is the memory card actually more expensive than the controller?
It might be advertising, but I'll allow it because it's so metal
But how did anyone figure out it was a SanDisk SD card? Card details were redacted.
There's only 3 manufacturers of SD cards in any volume, you can compare the branding and font choices and see who's it is.
Presumably because it looks identical to a Sandisk extreme pro 512gb, with grey boxes drawn over the logo.
The report has a heavily redacted interview with a submarine expert. Who directed Titanic and The Abyss.
They’re not good at redacting.
SanDisk is one of the big three on SD-3C/SD Association.. so kinda regardless of the MFG it's 'one of theirs' in a roundabout way.
Them sandisks need to be pretty tough to not vaporize whilst cooking your USB ports. Ever notice how infernally hot they get under large file transfers?
Completely killed one of my ports after 50gbs
but what about the Logitech controller?
What I also learned from this article is that Scott Manley is still on Twitter.