CNF/Residue/SPectral Subtraction Method

[Marc Hildebrant]

Group,

D (of dbanani) posted a method for very noisily songs where you use the CNF in a "unique" manner to find and remove impulse noise. While I have seen the you-tube video and read some descriptions of the method, I have to say that I really do not understand the approach. D, could you explain it some ?

Marc

[Craig Maier]

I do not understand it either. It seems redundant to use spectral subtraction in keep residue mode (which already subtracts the the entire signal from the noise signal) and then to go back and invert it and re-subtract it again. Mathematically, it seems to me that leaves you right back to where you started if you would have simply used spectral subtraction with the proper setting of the attenuation control.

Craig

[dbanani]

[FONT=arial]

Group,

D (of dbanani) posted a method for very noisily songs where you use the CNF in a "unique" manner to find and remove impulse noise. While I have seen the you-tube video and read some descriptions of the method, I have to say that I really do not understand the approach. D, could you explain it some ?

Marc

Hi Marc
This is the method I use for removing impulse noise from VERY badly scratched records with DC8.5 and DC10Forensics
This method is a last resort when nothing else seems to give satisfactory results, and I REALLY want a clean copy. I always prefer to buy or borrow a better copy...but sometimes there are no other copies.
It is dependent on the material, the types of damage to the record and the resulting clicks, pops, and crackle. Every record is unique and the settings which work for one will not be the best possible for any other. ALL filter settings are just a starting point and must be fine tuned to the material for best results.

This can be a time consuming process (aggressive filter settings on a 96/24 file take a long time to process) and each setting and checkbox for each filter may need to be adjusted to achieve best results.
I strongly suggest testing and adjusting filter settings for each record on a short 20-30 second portion of the file. I do this by copying a very noisy portion of the file and then paste to new file. Then I adjust my filter settings for this small file until satisfied and only then do I run them against the entire "ORIGINAL FILE"

There is no reason to save any of the files created in the steps (except of course the final "DE-CLICKED FILE") or to open separate windows. Just Run a Filter, Copy, Ctrl U to the appropriate place, Invert and Paste-Mix.

The steps are:

<> OBTAIN "ORIGINAL FILE"

1. I use a recording made with a Flat Preamp with no adjustments for RIAA etc. 96/24 Sample Rate/Resolution - Fast Edit Mode This is the "ORIGINAL FILE"

<> CREATE "CNF RESIDUE FILE" Method 1

2. Run the CNF on the"ORIGINAL FILE" with VERY aggressive settings(See A-D below) which will remove all the VISIBLE impulse noise in the peak file. The result is a "CNF RESIDUE FILE" which contains the residue of the "ORIGINAL FILE" after processing by the CNF

A. Do not be concerned if a bit of music goes with the noise since it will be removed from the "CNF RESIDUE FILE" later.
B. I start with Normal CNF, FFT size of 16384, Attack - 1, Artifacts - 1, Attenuation - 92, Artifact Suppression checked and Keep Residue checked
C. Use the bundled Preset Lazy Mans Noise Filter (Flat)). For LPs I tweak this by tapering down the portion of the curve below 400 Hz slightly and tapering up slightly above 6000 HZ producing an S shaped curve (thus some compensation for RIAA EQ)
D. Adjust the entire curve up or down using the Shift Threshold arrows until all visible impulses are removed when the CNF is run but no more than necessary for this removal.

<> ***WORK AROUND METHOD TO CREATE "CNF RESIDUE FILE" FOR STEREO ORIGINAL WITH FFT 16384 AND CHECK RESIDUE BOX UNCHECKED*** method 2

***(Note that steps 3-6 are not necessary if using a mono file or an FFT less than 16384 and checking the keep residue box.)
(Note also that DCForensics10.3 and DCArt10 will eliminate this limitation and checking the Keep Residue box will be the preferred method for all settings/files eliminating the need to perform steps 3 - 6 entirely)

3. Complete steps 1 and 2 above (with Keep Residue Unchecked on CNF) then copy the resulting file to the clipboard.

4. Return to the "ORIGINAL FILE" by using Ctrl U or the Fast-Edit History

5. Use the File Conversion Filter with both Invert Phase boxes checked (left and right) Stereo to Stereo and 0.00 Time Offset, Amplitude 0.0 for both channels. Run the File Conversion

6. Paste-mix the cleaned file back into the Inverted "ORIGINAL FILE" file using Past-Mix in the Edit menu . This results in a "CNF RESIDUE FILE" which contains the residue of the original CNF

<> CREATE A "MUSIC PORTION OF RESIDUE" FILE

7. Run Expert Impulse Filter ON THE "CNF RESIDUE FILE" from Step 2 or Step 6 using settings Threshold 135 - Size 30 - Tracking 0 - do not check solo/brass - Use Vinyl LP or 78 as appropriate.

8. If resulting file has much impulse noise remaining, run additionally run EZ Clean Filter with settings Scratch 95 - Crackle 95 - Hiss 20 - Hum 60 or 50 (depending on source of recording) or EZImpulse Noise Filter with Preset - Very Aggressive Impulse Filter. I call this resulting file from Step 9 "MUSIC PORTION OF RESIDUE"

(Note that if your final "DE-CLICKED FILE" from this entire process has some distortion, these settings are probably too aggressive and Step 8 is the place to start backing off on the agressive filter settings to achieve a distortion free result)

<> CREATE A "CLICK FILE"

9. Copy the resulting "MUSIC PORTION OF RESIDUE" file to the clipboard.

10. Return to the "CNF RESIDUE FILE" from Step 2 (or step 6 if NOT using the Keep Residue function of the CNF) by Ctrl U (Ctrl U 2 times if you ran two filters in step 8) or using the fast edit history.

11. Run the File Conversion Filter (settings as in Step 5)

12. Paste-mix the contents of the clipboard back into the Inverted "CNF RESIDUE FILE" - This will remove most of the music from the residue leaving mainly impulse noise (I call it a "CLICK FILE")

<> PASTE-MIX the "CLICK FILE" BACK INTO "ORIGINAL FILE"

13. Copy the resulting "CLICK FILE" from Step 12 to the clipboard

14. Return to the "ORIGINAL FILE" Ctrl U 2 times

15. Run the File Conversion Filter (settings as in Step 5) on the original file again.

16. Paste-Mix the "CLICK FILE" from the clipboard into the "ORIGINAL FILE".

(Note: If the peak file shows an increase in impulse noise instead of a decrease then Ctrl U - 2 times and paste-mix again. This happens when using the workaround method for creating the "CNF RESIDUE FILE".

<> THE RESULT IS THE "DE-CLICKED FILE". (Manual declicking a FEW impulses is occasionally necessary.)

If using DC10Forensics, I use the Spectral Filter in Spectral Copy mode to match the spectral response of the "DE-CLICKED FILE" to the "ORIGINAL FILE" resulting some improvement.

Then proceed as usual for final cleanup/enhancement.

This is a complicated method to lay out in words. But with DC8.5 I just completed the entire process above in about a minute and a half for a 38 second file. Of course these starting presets are already saved for each of the filters. Most of the minute and a half time was waiting for filters to complete processing.

Remember, there is no need to save any of these files (except of course the final "DE-CLICKED FILE") or open separate windows. Just Run Filter, Copy, Ctrl U to the appropriate place, Invert and Paste-Mix.

D

[Craig Maier]

Does this method work with the CNF Spectral Subtraction mode?

Craig

[dbanani]

Hi Craig
I just did the entire process on a noisy 44.1/16 file I had up on DC8 and it worked in Spectral Subtraction mode but did not remove quite as much impulse noise as Normal CNF did using the same settings for all filters and 8192 FFT
Tried it again on a 96/24 file with 16384 FFT and the results in Spectral Subtraction mode are very similar to those with Normal CNF
So yes, this works in in Spectral Subtraction mode also
D

[Craig Maier]

I am having the same problem with copy and paste too on the Forum too! I had been trying to show the new features and bug fixes for DCForensics10.03.

Craig

[dbanani]

Yes Craig
It looks perfect until I click save then the post gets truncated when it appears on the forum
I have been trying for two hours and haven't been able to post the steps in one post yet.
UPDATE - Just got it!
The problem appears to be in Microsoft word when using a hyphen (-) Word is changing some of my hyphens to a longer version of a hyphen. I did a search and replace for this elongated hyphen and the post worked as expected. This may be because I started the post by typing into the forum and then decided to copy my draft and past it into Microsoft Word.
Word may not be recognizing the hyphen symbol the forum uses.
Once I changed them the Steps posted as expected.
D

[Craig Maier]

Thanks dbanni. I will give that a try.

[Marc Hildebrant]

D,

Thank you for your long post on the "noisy" process. I have a couple of songs that are really bad and would like to fix up.

Marc

[dbanani]

You are welcome Marc, Let me know if you have any questions.
The process is really pretty simple once you get the hang of it. Finding the right filter settings is really the only difficult part.
D