78 Rpm Restoration Questions

[Doug]

Craig & Rick, these are some questions that I'd like you to answer, if possible, to solve my curiosity!

1. If the weighting control in the "median filter" is set to 0, does it operate essentially the same as the "averaging filter" ? (I'm assuming yes - dumber Q to come).

2. With the "Dynamic Noise Filter" one can set the "Filter Frequency". Does this filter operate by only reducing the gain of sounds/frequencies above the "Filter Frequency" or is it across a specific part of the spectrum set by the attack & release sliders?

3. I still struggle with this one! With the CNF, when trying to remove hiss AND conserve (as best possible) what is left of the fidelity and "crisp" sound of a 78, what FFT size should one choose? ie, 512 - but I think this makes the sound more "rounded", whereas 4096 makes more digital artefacts?

I look forward to your knowledge.
PS. Still no luck with denoising VDL yet!

[Craig Maier]

The Median Filter never calculates average or moving average, no matter where the weighting control is set. Median is the centered value in an order grouping of integers. See the users guide for details on the math of this function and compare it to the average filter function.

The Dynamics Processor can increase or decrease the gain applied to a signal. When in normal mode (non enhancer mode), the signal is decreased below the threshold value. When in enhancer mode, the signal is increased in amplitude above the threshold value. The amount of increase or decrease is determined by the HF Gain Control setting.

As for FFT size settings for the CNF, there is no good rule of thumb to use. Fast moving music with lots of transients will benefit from smaller FFT sizes and faster attack and release times. Slow moving music (Waltzes, etc) will benefit from larger FFT sizes and slower attack and release times. You really need to experiment. I think that the key for success with this filter is not to try to remove every last bit of noise. If you do so, you are going to get artifacts. I always satisify myself with a reduction, but never to zero noise levels. So, the attenuation control is very important to me. Sometimes, I will actually adjust the "blue touch points" and move them around in the 5 to 20 KHz range (sometimes adding more touch points) to improve the performance of the system. The system allows you to add a lot more than the default value (see the users guide for the actual number - I forget what that number is).

One last hint on the CNF - sometimes the smoothing checkbox will reduce the amount of artifacting that will occur at a given level of noise reduction. Again, this is material dependent and must be determined by experimentation.

I hope that this helps.

[Doug]

Helps? Almost!
Is there an advantage in using the median over the averaging filter? The averaging filter appears to do a good job at reducing usless hiss and crackle without taking out the highs...The median I don't quite understand where to put the weighting for a good 78 (roughly what weighting would you suggest for a good 78).

2ndly, I run most of my 78s through a 6-8kHz low pass filter before using the CNF. I'm assuming that the averaging filter probably does a better job when put on 3-4 samples. Do you think it wise to forget about the low pass and just average out HF noise instead to get the same affect?

To anyone who hasn't tried the averaging filter, I strongly recommend giving it a go on all 78s

[Craig Maier]

The Median Filter and the Averaging filter produce different results. The Median Filter tends to eliminate large signal changes (high slew rates) while maintaining the overall small signal response of the system. The averaging filter behaves somewhat like a low pass filter but uses a different technique and thus sounds differently than a low pass filter.

That probably did not answer your question. On 78's, I use the Median filter set to 3 or 5 samples with the weighting control set to the middle. The weighting control is designed more for Forensics audio applications wherein the signal is very garbled (having no annunciation of the siblant sounds) thus allowing the system to re-insert these signals for intelligibility purposes. I use the Averaging filter set to the range of values that you suggest to reduce surface noise on 78's.

I hope that this helps.

[Doug]

Avg, Median, VVA

Ahh! So use both.


Median set to 3-5 samples (or more for louder crackling) and weighting of 0,
Averaging set to 3-4 samples (higher for smaller bandwidth recordings)

What I found confusing is that the 78rpm preset in the median filter has the weighting preset to 100. As a result, I assumed it should always be 100 for shellac recordings and never got good results with sample sizes greater than 3. As for the averaging filter, I see that it leaves behind more of the high-frequency transients/harmonics than a low pass filter (quite obvious when looking at the spectrogram).

I still tend to have variable success in denoising 78s. I'm trying a new method (involving the avg filter). But, as per usual, what works well for one can fail dismally on another! I find it a little aggravating. What I find interesting, is that some electrical recordings have much greater bandwidth than others. I have a few that run up to 15kHz (at best) whereas another may only just reach 6kHz. The latter sound dull and require enhancement with the VVA (another challenge of mine!). I think the bandwidth is where my problems are arrising.

I don't suppose you can throw some tips on the VVA to excite those missing harmonics? From memory, the harmonic exciting presets on the VVA make a 78 restoration sound pretty rough/heavy. What settings for drive, mix etc would you suggest to start with?

I must say that I am extremely happy with my successes thus far. As for the few... is it that some recordings are just a pain in the A$$ no matter what you try?

Thanks again Craig. If I can make something that works on the 78s suffering ADD, I'll post it on the exchange.

[Craig Maier]

The 78 Median Filter preset that you refer to has a bug. The weighting should have been 0.


Low bandwidth recordings make denoising difficult because the noise and the HF signal are often at the same absolute level. On very noisey and limited bandwidth 78 RPM records, I tend to use the Auto Spectrum CNF lightly applied.

As for the VVA, the balancing act is to brighten things up but not bring back too much noise. Thus, I usually use a fairly non-linear RF type mixing tube like the 12AT7 and I keep the "Detail" Control low or all the way down and use just the mixing control to achieve the desired result. The drive and operating point are usually set around in the middle of their operating range.

It is tricky business when you are particular about your result, but you do have the tools there to do the job. Lots of measurement capability as well as lots of control over all of the systems paramaters.

And, yes, almost every old 78 is different; wear, age, brand, and recording technique all play into what you are working with. I wish that I could give you a magic arrow, but I can not. 78's are tougher than vinyl by far, as you have seen.

[Doug]

Thanks Craig,
I'll give the above a shot, I think that is close enough to a magic arrow as one could possibly ask.

Your help and thoughts are greatly appreciated. as always!

[Doug]

CREATIVE Ideas for using the CNF.

Craig, I think most of my problems stem from using the Median and CNF all WRONG! Your idea earlier of starting with a low attack and slowly increasing it on the CNF in "keep residue" mode works well. Doing this I have only just realised that it leaves behind more of the transients while reducing the noise. Up until now, I have never used the attack and release settings correctly.

While on the topic, here's a couple of "interesting" CNF methods I find that work well (even with incorrect CNF operation!) which others may like to try on 78s when despirate :

1. Multiple CNF passes with different FFT sizes
2. Frequency specific CNF passes with different FFT sizes.

Always Ensure that you have EQd, Average filtered (& or Median filtered) the audio before using the CNF. I also remove as much distortion as possible using the Dynamics processor on the whole file and then selectively on bad spots. This step is important as it prevents distortions from inducing digital artefacts in the CNF stage. It also removes the hiss/swirls in the loud passages.

As for the CNF, I don't know how many times in the past people have mentioned competitors restoration software. One in particular uses a CNF with a spectral decay rate (Eqv to release time?) and a transition setting for noise removal. I'm sure people have given trial versions a go - and probably got crappy results - it is hard to set it up and get it correct. Well, you can use the CNF in DC-Art in either a single pass to remove noise (if you got a great record/skill!), or in multiple passes, which I would equate to this transitional noise removal. I start off with a high FFT (say 4096 or 8000+ for really bad record) and reduce the noise by 4-5db. Then FFT 2096 the same followed by 512 focused on high frequencies with more attenuation to remove the hiss. 3 FFT steps is usually enough. With this method you can CONTROL the noise removed, the transition rate with which noise is removed, and retain any fidelity by removing final hiss with the 512 FFT. Less artefacts, better quality and more FORGIVING than a preset transition rate! Works great for some bad 78s.

A second method I've only just started trying:
Rumble, surface noise and hiss are all different between records, have different properties (ie frequency, time responses) and are not always of equal volume on a record. Since the CNF in DC allows us to target frequencies ? why not?

I first removed rumble by setting the CNF at 4096-8000+ and moved the blue markers above the red noise line in the region of 0-500Hz or less, all other markers I placed bellow the noise threshold. Setting the attack to something like 100+ms and release to ~200ms, you have a selective rumble filter that can reduce rumble by up to 50dB without any obvious audio degradation! But be careful not too lose soft bass notes.

Midrange surface noise, set the attack to around 50ms+ (using Craig?s method of the CNF in keep residue mode). Make the CNF FFT 2048-4096, release time double the attack time number. Sample the noise then simply move the blue inflexion points above the red noise threshold line for frequencies around 500Hz+ to 2-3kHz, all others get pushed bellow. I usually use a curve where I cross over the threshold line. The idea is to remove surface noise but not the hiss or the high frequencies. I also ensure I cannot hear any digital artefacts as a result of where I cross the noise removal with the noise threshold. At best, I would attenuate the noise by 8dB or more for an average 78.

I end up with hiss and faint surface nose, nothing else. Follow this up in the same way using an FFT of between 512 and 1000, attack can be as low as 30, release ~80-90. You should be able to reduce the remaining noise and hiss anywhere up to 15dB if you wish to have no noise in the background. An alternate, but equally good final step is to reduce noise by 5-8dB (not 15dB) and remove additional hiss/noise with the dynamic noise filter (this gives a faint noise in the background). These last steps help avoid artefacts and the loss of high frequencies while suppressing hiss in quiet passages.

Long I know, but both methods I find are FORGIVING and give a nice ?rounded? sound with a touch of noise in the distant background (if you want it there)... Both methods also maintain a lot of musical detail. You can always go back and attempt different stages of noise removal if the final product sounds digital. I suggest if people are having difficulty with some records (like myself) a little creativness in the CNF stage can take a person and restoration a long way. Despite some previous discouragement of multiple CNF passes, it does work...I don't think there is right or wrong way, just what works for a particular record.

My methods are a little complex aren't they Craig?

[DougMac]

Doug,

Thanks for sharing your experimentation methods and the results you've achieved. I'm looking forward to trying them myself. Don't worry about being long, the detail is well worth it!

Doug

[Craig Maier]

Quoting Doug:

"My methods are a little complex aren't they Craig?"

-------------------------------------------

But, it is hard to argue with success! As you point out, there are lots of ways to skin this cat and you have found two. Thanks for the carefully articulated explanation which I am sure others will be able to use.

[Doug]

Thanks Doug & Craig....I can guarantee it works at my end - the 1st method has for 2(?) yrs. As for the 2nd method, sometimes it works if you separate only bass and trebble (CNF for each) and use a higher FFT like 1000+ (I can't remeber if it is 1012 or something) to do the midrange and treble together. A little more noise can get through on a bad recording, but it retains the authenticity of the original recording. Just a way of getting a new sound with a "touch of old".

As for removing distortions, I use the dynamics processor in a similar manner to the 45rpm distortion preset. I change the threshold to ~-12 to -22 dB depending on the loudness of the recording and the amount of distortion. You're best judging this from the peak/wave file display. Then change the ratio to 3-4 (as low as 2-2.5 if the distortion/swish is slight). Attack is set as low as possible then adjust the Frequency for somewhere between 3500Hz up to 5-6000Hz for a good modern 78rpm record. Lower Freqs remove more distortion in audio, while the higher freqs remove hiss/swirls etc. The main idea is to make the audio above say 4-5000Hz sound as even as possible across the entire recording despite loud/soft pasages etc. Otherwise, these sudden changes in noise vol cause swishing/digital artefacts and noise/hiss to escape both the CNF and Dynamic noise filter. Run that across the whole file, but try to maintian the fidelity with the setting chosen. always set these parameters with the fidelidy of the original in mind. Then, go back and use the same process selectively on bad spots - here be more aggressive with the Frequency & ratio of compression. I personally prefer to hear a loss of fidelity than my speakers rattling wildly with ugly distortions.

How do you guys normally attempt to reduce needle invited distortions? It's always the best records with the best songs that are heavily distorted...popularity results in lotsa use. I would like to hear other methods if possible.

[Craig Maier]

Quoting Doug: "How do you guys normally attempt to reduce needle invited distortions? It's always the best records with the best songs that are heavily distorted...popularity results in lotsa use. I would like to hear other methods if possible."


---------------------------------

I track distorted recordings very heavily. That seems to help. Sometimes, I play the recording wet which also seems to help. Lastly, I use your method found in the Dynamics processor or sometimes I use the top two bands of the Punch and Crunch effect. The Polynomial filter found under the Forensics filter can also be helpful, but you will need to learn a bit first to understand how to use this. It is a useful but tricky tool not to be used by the faint of heart.

[Lord Theo]

Creative Ideas For CNF for 78RPM

Hi Doug,
Like yourself, I've been experimenting in CNF by targeting low-midrange-high frequencies individually for noise removal. I tried your specs & found it very effective most of the time. Sometimes, a faint, but constant flanging sound remains in bkgrnd. Has this happened to you?

Regards,

Theo

[Doug]

Hi Theo,
As with most methods, sometimes it works at sometimes it doesn't...
I just found that I could pick "ball-park" attack and release times and still get exceptional results for my over-played 78s.

I'm not sure what you mean by constant flagging, but I interperate it as digital artefacts in the background??

When I get this I try removing the noise from 1-5kHz or so using an FFT of 4096 and set the attack a little slower (maybe 85ms - again, check in "keep-noise" mode for too much music contamination). Also, with the higher FFT you can push the blue threhold higher and try to pin point the exact frequencies where the flagging is occuring. I often do this with the Attenuation control set high (40+db) so that it is obvious when I get the responsible frequency band. Then, drop the attenuation down to a point where the noise is only just audible and not inducing obvious digital artefact. I find it helps to turn up the treble on the sound card controller or amplifier when trying to pin-point the frequencies. Run that filter. This step is important as you are trying to flatten the noise across the spectrum and make it more even. Sometimes it also helps to closely map the blue threshold line with the red theshold line from 5kHz onward so that part of the his is removed in this step...but it is important not to induce artefacts (you can try with smoothing on and off). it is difficult.

Then, without re-sampling the noise (works with/without resampling noise) change the FFT to 512 (I think 256 doesn't work all that well). Reduce the blue noise threshold line below the noise for 0-1kHz, then push the rest above the line. Using an FFT of 512 on the rest of the spectrum (1Khz onward) will result in noise removal under normal music conditions, but allows fidelity to excape around transients and loud passages much better without inducing artefacts. Lately I have run this at around 5-8dB attenuation (again with my treble control on the stereo maxed out). You should only be left with a faint hiss or less when done correctly. Sometimes the blue threshold line needs to be pushed 5-10db above the red noise line - in this situation, a lower attenuation usually works best. Other times, if the noise is very consistant, you can almost map the red noise line with the blue threshold line and attenuate the noise by 15db (none left at all!). It's give and take, but I think fidelity usually comes with some noise escaping.
You could always try an additional filter step of an FFT size of 1012 before the last 512 run. Each drop in FFT size captures the noise less precisely and rounds the noise response as you go...but the more steps, the more careful with the amount of attenuation you must be.

Lately I have tended for faint hiss, then running the outcome through the Dynamic Noise filter set to a threshold of 600-800, frequency of 4Kz-5kHz, short attack of ~12-20ms to release of 50-60ms. The lower the frequency of noise left, the longer the attack and release times need to become (up to 40ms and 80 respectively). If the sound is a little dull, try running the music through the punch & Crunch filter focused on the 3rd band (drop the threshold right down, but not into the hiss). Set the ratio to 2.0-3.0, attack to 55-75ms, release to ~1.2s and I tend not to attenuate the output at all. Sometimes it may also work better in the "narrow band" mode using either or the 3rd and 4th bands only and an expansion ratio of 1.3 to 1.4 (very low). Sometimes this works best before or after the Dynamic noise filter.

Long winded again - and unfortunately, I'm not an expert. I do this for the fun and enjoyment of hearing things that have been long forgotten. The stylus type chosen, the brand of 78, the age, the wear all affect the outcome. They vary so much and sometimes cannot be helped. I think the initial steps prior to the CNF are of up most importance as they set the best possible bench-mark for the remaining steps. I have tended lately to filter all my recordings off at 12kHz with a 12db/octave low pass filter following the averaging filter set at 3 samples...this tends to help reduce possible artefacts and limits the top response to ~14kHz at best. Old worn records I find can benefit from a Median filter (set at 3-5 samples) in addition to the Averaging filter (set at 4 samples) and a low pass of 8kHz 12db/octave filter. It limits bandwidth, but according to the spectrogram, it doesn't remove realistic content.

On that note, sometimes in noise removal on the CNF it helps to look at the spectrogram to see where the noise is focused, and how much damage the removal has caused to the remainder signal. Certainly use it to remove any remainder clicks and pops BEFORE using the CNF.