Sunday, December 7, 2014

3.5" floppy disks and Word 6

I thought I had seen the last of 3.5" floppies until last week.  A customer turned up on my door step with 4 3.5" floppies, and wanted back documents she had written 20 years ago.

Fortunately I still have a few PCs with internal floppy drives (on XP boxes).  These I could see were DS-DD 720K rather than the normal 1.44MB disks.  The customer mentioned an Apricot system, very popular in the mid 80s, but not totally PC compatible.  On placing the disk in the PC, it suggested a reformat, as the disk was not recognised.  Fortunately on examining the sectors it was clear that the disk was still sound, with not obvious sector errors.

Trying to read it was not an instant success.  It was FAT12, but as there were no subdirectories the CnW automatic parameters program failed.  The next stage was top fire up an old copy of InterMedia for Windows.  Being a 16 bit program, this had to be on an XP box.  InterMedia for Windows has a routine called MS Auto that will try and determine disk parameters automatically for floppy disks, and with this the directory was soon visible.

I could now read the files, a mixture of .DOC and associated .BAK files.  However, to be expected, trying to open the .DOC file indicated it was from a Word 6  system.  Word 2003 is meant to have a filter to read this, but it would not work (mswrd632.wpc) so the files could be seen, but with a lot of binary code included.  I am not sure that Word 2003 is supported anymore, being an XP era program.

The next stage was to use InterMedia for Windows again that has protocol handlers for hundreds of old word processing packages, including Word (several versions).  This then  produced nice clean text, with embedded  commands (IMIC2) for some formatting.  These commands were then stripped out using translation tables to generate a simple text file that could be read on a modern PC.

InterMedia for Windows now looks like an old program - it is over 10 years since I last updated it, but it did produce a nice clean result.

Friday, November 14, 2014

Windows 10

I have an oldish PC, bought originally to run Vista.  This I then updated to 7 and 8.  This week I download the Microsoft Technical Preview, otherwise to be called Windows 10.

The installation required DVD to be burnt and then the occasion button pressing while loading.

The final loaded PC was almost exactly as I had it setup in Windows 8.  All programs ran, in particular CnW Recovery.  ie the upgrade could not have been more painless - although it is reported that come April, it might all stop working.  It even worked as a remote desk top with no additional attention.

I cannot comment on performance as the PC is an old Core Duo but it does have 8GB RAM.  I am currently using it on video file recovery testing and after a few days it has just worked.

The famous start menu is back, with a few tweaks, and a bit of the metro style start at the same time.

The only downside is that Norton 360 does not work, so the system is protected by the free Microsoft Defender.

So far, so good.

Friday, August 22, 2014

Viewing H264 raw video files

This is a very useful link I would like to share.

Often one has a video file that one cannot view - maybe from a CCTV system.  It is quite common to be H264 or similar, but does not have a standard wrapper, eg .MOV, or .MP4.

The very good free program VLC does have a feature to allow you to do this.  The following Youtube link does describe how to

As a very quick summary, the following may help

  1.   Tools
  2.   Preferences
  3.   At the bottom select 'All'
  4.   In Stream output  select 'Demuxers'
  5.   Then select the  Mux module  - (Automatic is the default)

Monday, August 11, 2014

Companies that promise something they cannot provide

I am not going to name names, but recently I have several reports from my satisfied customers moaning about other companies who have taken their money for software that cannot do what they say.  The main focus is on recovery of video deleted from the camera memory chip.

As I have written here several times, when video is recorded in the camera, the physical sequence of data is different to the logical sequence, the video is written in fragments, but the FAT keeps track of the logical order.  When the file is deleted, this sequence is also lost.  Recovery is complex and cannot be done by simple data carving.

If a video is written to a hard drive, it will normally be sequential and thus a data carving approach to recovery will often work.  I believe that software companies just test hard disk deletions and say it will work.  I have tried a few demo products with my 'favourite' GoPro camera images.  They fall flat on their face.  The software will report files, that look the correct size, but will never ever be viewable.

From CnW Recovery point of view, I have an uphill struggle (on a low budget) to let potentials users know that there is a product that will have a very high success rate of recovery of MP4, MOV, AVI, MTS videos that have been deleted from the camera memory chip.

It is a shame that the internet has to boast and boast even when something does not work.  At CnW, not everything works, but there is always free backup to resolve the problem - fairly quickly.

Tuesday, April 8, 2014

Apple disk that was a FAT32

I see a fair number of Apple disks and most start with sector 0 having the letter 'ER' in the first two bytes.  Others have a GPT boot sector.  They then always have an HFS+ file system.

I have just seen a new variation.  It started with a normal ER boot sector.  The next sector started, as normal with PM and was the Apple Partition map pointers.  The third sector then very unusually pointed to DOS_FAT_32.

First attempt to read the disk with CnW failed, but with a few manual inputs it was possible.  Firs I had to set the partition type to be FAT32, and the partition start to be 0x40.  After this, the disk read OK with CnW Recovery software.  Due to hardware issues (the reason I had the drive in the first place) I could not try the drive on a MAC, so I do not know if a standard MAC would read the drive or not.

A future version of CnW will understand this type of drive automatically.

Wednesday, January 29, 2014

Deleted GoPro video with high and low resolution

MP4 video is often recorded out of sequence.  This is because when the recording starts there is no knowledge of the file size.  Thus the meta data is added physically at the end of the physical file, though logically it is stored near the file start.

CnW can process this type of deleted file.  A new variation found in one version of the GoPro Hero 3+ Black camera is stored both high and low resolution video.  In this case, the video is multiplexed so physically on the disk there will be sections of high resolution, followed by low resolution.  The programming challenge is to determine which cluster is part of which video stream.  By examining the meta data this is possible and current results are very encouraging.  The next stage will be to ensure there are no glitches in the recovered streams

Sunday, December 29, 2013

Recover deleted MP4 files from Sony PMW-F3 update

My last post reported good progress on recovery of Sony PMW-F3 files.  I am now pleased to report some extremely good results.

The final approach involved scanning the complete disk for possible audio and video elements.  As this is fairly machine intensive it was very satisfying from the programming point of view to make use of parallel processing.  While one cluster was being read from the physical device, the previous cluster was examined for audio data, and in parallel, video data.

Once the disk has been scanned it is then quick to isolate any pre scanned video structure with it's location within a cluster.  On a small memory device one is normally luck that each structure will have a unique offset.  For large memory device there will be duplicate entries.  In these cases the physical location of the matching cluster will be used to determine the correct one.

When a video file is deleted it is normal for the data to remain, although may well be non sequential.  The critical section to find is the moov atom.  This contains pointers to frame starts and audio buffers.  By tying up the moov pointers with the physical clusters a valid file can be reconstructed.

Simple?  In some respects yes, but recognising patterns is not a simple computer task.  It is very much an exercise in fitting together the best matches.  It is a bit like doing a jigsaw that has no picture.  When it works, the results are fantastic.

Many videos are recorded in two sections, the Sony sometimes uses over 100 sections, so joining these together is a great success.

Sunday, December 1, 2013

Sony Video Camera PMW-F3 deleted MP4 recovery

A major and on going CnW Recovery development is processing fragmented video files.  In particular these are ones that have been deleted on the camera.  When a FAT32 file is deleted, the allocation table is cleared down and so the order of clusters is lost.  For may applications, this does not matter as the file is sequential and so can be recovered - and this is often true for photos.  Videos however tend to be long and an MP4 /3GP / MOV file has three main sections (or atoms).

  • A short 'ftyp' header
  • A mid size 'moov' that stores pointers
  • A big 'mdat' that stores the audio and video
Logically the sequence is normally either ftyp-moov-mdat  or ftyp-mdat-moov.

When recording it is impossible to know the length of the moov or mdat atom.  For this reason the mdat is stored on the memory chip, and often the moov is stored in the camera RAM until the end.  On finalisation it is then written to the memory chip.  If the required sequence is ftyp-moov-mdata some 'clever' fiddling of the FAT is performed by the camera to make the file logically sequential.

The CnW program has been developed to handle the above for many video types but then  the Sony PMW-F3 format was found.  The big difference this time is that the 'mdat' is stored in numerous chunks, and not always in sequence on the memory chip.  The challenge that CnW is working on is to find these chunks and reconstruct the video.  This is performed in several stages
  • Chip is scanned for all ftyp, mdat and moov headers
  • Chip is scanned for all MP4A audio clusters
  • If required a fragmented moov is reconstructed
  • The video and audio frames are located based on offset within a cluster.  Special routines are required when there are multiple audio or video frames stored at the same location within a cluster
  • Often the frame pointers point to clusters several clusters later than the previous one. In these cases the gap between the known cluster locations has be filled in by working forward and backwards from known good locations
The current results are very reasonable, but still more tweaking is required.  However one customer reported that CnW recovery does recover viewable video, and no other program got anywhere near this. 

CnW is very sceptical of many adverts that claim video recovery - it may work from a hard disk, but CnW has major doubts about working form camera memory chips

Monday, September 16, 2013

GoPro Hero 3 recovery of deleted files

I have often said of data structures, 'if it can be done, it will be done'.  ie almost anything is possible, and so expect many variations.  In the words of the 'Hitch Hikers Guide to the Universe' - Expect the unexpected.

A recent view of a GoPro Hero 3 camera confirmed the above.  As I have written in the past, video camera often record the data not in a different physical to logical sequence.  The slight of hand is that the FAT defines the logical sequence.  Thus many camera record the video data physically earlier than the file start sector.  On the GoPro Hero 3 camera another twist has been discovered.  Two video streams can be recorded at the same time - a high resolution, and a smaller low resolution.  The physical sequence on the disk could be as below

Just to complicate the above, there can also be jpeg thumbnails, and text status files inserted in the above stream.  Standard data carving is totally useless as nothing is in sequence.

The CnW MP4/3GP Wizard works hard, but will recover, with a high success rate videos that have been deleted.

Tuesday, September 3, 2013

Windows 8

It was time for a new PC and so Windows 8 - 64 seems the obvious option in order to keep up to date.  I ordered reasonable spec system (3.6GHz, Core 7, 12GB RAM) and a week or so later it arrived.

I have had Windows 8-32 since it came out, but not as a main PC.  This post will describe some of the issues I've had, and some solutions.

My work (data recovery) requires lots of disk space.  I keep the main 2TB system drive as the system and development drive.  All customer data is stored on other drives - or a new 9TB NAS RAID.  My first job was to add a second 3TB drive to the PC.  This was a reasonable quick job with the screwdriver, and the drive was seen in the BIOS.  However, what ever I did, I could not see it in Windows 8.  Google to the rescue and it turns out that Windows 8, being much more secure will not recognise extra hardware (though it could see USB drives).  The solution was a BIOS option switch to remove the 'Secure boot' option.  Drive now seen, and the secure boot can now be re-enabled.

The next problem was Norton antivirus.  The PC came with McAfee, but the rest of my systems all have Norton, on a group licence.  I uninstall McAfee and installed Norton 360 but then problems began.  Various problems, mainly IE10 and other internet related products had issues.  Worse still, each time I rebooted the PC, Norton stopped working and often came up with the error 8504,101.  Google had several ideas for this, including running NPE.exe, uninstall and re-install.  I tried many of these suggestions, but none worked.  The error message is common on Google posts, and so I installed AVG and started to get some work done.  Even the Beta version of N360 had the same problem.

On hardware, I have become a complete fan of multiple screens.  The new PC has 3 video outputs,  but 2 of them are HDMI, and I only have (old) VGA screens.  I hope the eBay adaptor will arrive soon and let me work with 2, or maybe 3 screens at once.

A final issue I had was over a security dongle.  The drive is not signed, and so Windows 8 will not load it.  The solution was the result of another search and involves booting Windows in a mode to accept unsigned drivers.  The instructions (with due credit to someone else) are as follows

1. Windows Key + R
2. Enter shutdown.exe /r /o /f /t 00
3. Click the OK button
System reboots here

4. System will restart to a Choose an option screen
5. Select Troubleshoot from Choose an option screen
6. Select Advanced options from Troubleshoot screen
7. Select Windows Startup Settings from Advanced options screen
8. Click Restart button
9. System will restart to Advanced Boot Options screen
10. Select Disable Driver Signature Enforcement
11. Once the system starts, install the drivers as normal


Looking forward to Windows 8.1 where I hope the metro screen will be less important, though I will start playing with some of the apps.