Last update: December 1st, 2008
(broken links fixed)
Table of contents
Introduction
Scheme I - "blind watermarking" scheme
Scheme II - "strict watermarking" scheme
Third-party assessments
Contact information
I've created this page to provide some information on a watermarking technique I have developed. The technique is currently US patent pending (patent application 60/907,778).
I propose a watermarking method that exploits an idea of signals shifting in the time domain. I will not disclose here the whole description of the invented method, but I'll briefly explain the idea behind it and will provide two different implementations of this idea together with real PC applications and sound examples. I believe this technique maybe potentially interesting both technologically and commercially.
Signals shifting in time is very simple and at the same time very informative operation. I've noticed that this method can be easily used as a basis of watermarking technique. My idea is to encode a binary watermarking data within a carrier audio signal in time domain by time-shifting the carrier signal blocks in full frequency band or in frequency sub-bands. Below I present two different realizations, and actually two different approaches to this idea.
If you're interested in practical examples only, you can skip reading the information below and scroll the page down to the headlines "Example implementation", Scheme I or "Example implementation", Scheme II.
Two approaches - two schemes
Various different watermarking techniques can be developed based on the idea of signal blocks shifting in time. Actually, not only digital, but also “analogue” watermarking exploiting the same idea is possible. At the same time all possible implementations of this idea are eventually converge to two principally different approaches:
- the first approach implies that the source (unmodified) data is needed to extract watermark from the watermarked signal; this approach is generally called "strict watermarking";
- the second approach does not require the source data for watermark extraction; this approach is generally called "blind watermarking".
Both of the approaches have their pros and cons, and are suitable in different applications. Taking the initial idea as a basis, I've developed my own preliminary implementations of the two approaches. I propose two different watermark embedding schemes, both based on the idea of signals time-shifting described above.
Scheme I - "blind watermarking" scheme
The scheme is based on the signal time-shifting idea briefly described above, and exploits "similarity" of audio data in two channels of a stereo signal. This watermarking scheme does not require the source audio signal for watermark extraction.
Briefly on fundamental features and restrictions of this scheme:
- No need in source (unmodified) signal to extract the watermark;
- Two (stereo) or more signal channels are required to embed and extract the watermark;
- Imperceptible watermarking;
- Robust to sample rate conversion, re-quantization, transcoding (including psychoacoustical algorithms like MP3, OGG Vorbis, etc.), wave manipulations (cutting, shifting in time, splitting), spectral manipulations and effects processing;
- Scalable watermarking data capacity (depends on desired reliability and embedding parameters);
- Depending on implementation, a special “private key” may be defined as a requirement for watermark extraction.
Target applications:
- web-based on-line music and media content stores, or any other content distribution;
- audio recording and production studios;
- broadcasting companies.
Below is my demonstration implementation of this watermarking technique. This implementation is written on MatLab and is available in a form of stand-alone executable application for Windows. It is relatively slow (compared to how fast it could be being implemented on C/C++) as it was developed for demonstration, research and evaluation purposes only. On the other hand, it is fully functional watermarking solution, capable to demonstrate abilities of the technique.
Several watermarked signals together with watermark extractor (decoder) are available for direct downloading on the page below. Watermark embedding utility (encoder) is provided upon request.
A number of key notes on this particular implementation of the scheme:
- Operates with stereo PCM files 44.1 Khz / 16 bit; applicable only for stereo audio data (having stereo sound image);
- Watermarking is performed in frequency subbands providing imperceptible watermarking by the human ear;
- Special “private decoding key” is created during watermark embedding. Watermark extraction is impossible without this key. The private key is individual for each particular source signal and depends only on watermarking data size (irrespective to its contents). It means that private key is unique for each particular source signal, but does not depend on watermarking data contents. In this particular implementation private key size is around 5 Kb of data per 1 minute of watermarked audio signal (resulting in around 20 Gb of data for 1 million musical tracks with average duration of 4 minutes);
- Demonstrates very high robustness to spectrum and wave manipulations (including signal splitting, cutting, padding), sample rate conversion, re-quantization, effects processing and transcoding. For example, the watermark survives successfully under multiple subsequent transcoding cycles using psychoacoustical subband coders;
- Watermarking payload size can vary from ones to tens of bytes; watermarking “aggressiveness” (density) is a parameter. It's worth noting that for a specific signal and watermarking density, the smaller payload size is, the higher the robustness.
I do not provide the encoder here, it is available upon request. Instead, I provide several audio examples of source and watermarked audio signals, and the watermark extractor to play with (see below).
Source audio signals (WAV PCM, 44.1 Khz, 16 bit, stereo, 50 seconds long, 8820 Kbytes)
bach
bachms
gazebo
jarre
We will encode these four audio signals with the following watermarks:
WM1 = 0x01234567 (4 bytes)
WM2 = 0xABCDEF123456 (6 bytes)
WM3 = 0x0987654321AB (6 bytes)
WM4 = 0x0123456789ABCDEF (8 bytes)
Note: for the sake of example, let's note that as WMKey2 and WMkey3 are of the same length, they will result in the same private key for each particular source signal. It is just to show that if used watermark is always of the same length, then private key is always the same the particular audio signal, no matter what watermark payload cotents are.
The table below contains a list of downloadable files obtained by embedding each of the above watermarks (WM1, ..., WM4) into each of the four test signals. Private keys for watermark extraction are also provided.
Source file |
Used watermark | Resulting watermarked file |
Watermark payload size / density |
Private key-file for watermark extraction |
| gazebo.wav jarre.wav bach.wav brahms.wav |
WM1=0x01234567 |
gazebo_out_wm1.wav |
4 bytes / 0.75 6 bytes / 0.75 6 bytes / 0.75 8 bytes / 0.75 4 bytes / 0.75 6 bytes / 0.75 6 bytes / 0.75 8 bytes / 0.75 4 bytes / 0.75 6 bytes / 0.75 6 bytes / 0.75 8 bytes / 0.75 4 bytes / 0.75 6 bytes / 0.75 6 bytes / 0.75 8 bytes / 0.75 |
gazebo_key_(4_0.75).mat |
To show the robustness of the proposed scheme, I put four files that were obtained by transcoding of the watermarked signals from the table above with a long chain of encoding and decoding stages, namely: Watermarked WAV -> MP3 256 Kbps -> MP3 192 Kbps -> MP3 192 Kbps -> MP3 128 Kbps -> MP3 128 Kbps -> MP3 128 Kbps -> MP3 128 Kbps -> OGG 128 Kbps -> MP3 128 Kbps -> OGG 128 Kbps -> resulting WAV (a batch-file used for transcoding is here):
bach_out_wm1_transcoded.wav
brahms_out_wm4_transcoded.wav
gazebo_out_wm3_transcoded.wav
jarre_out_wm4_transcoded.wav
In addition to the presented audio examples, I provide the watermark extractor (decoder) which you can use to extract watermarking data from the example audio files presented above, and check out the results. Of course, you can also examine the robustness of this watermarking scheme by applying different audio transforms and effects, and trying to decode the watermark from the transformed files.
You can download the watermark extractor here (win32/64 application):
wm_stereo_decoder_x86.zip (Windows x86 executable, 6 Mb, ZIP)
wm_stereo_decoder_x64.zip (Windows x64 executable, 6 Mb, ZIP)
Watermark embedding utility (encoder) is available upon request.
Note: You will have to have MatLAB Component Runtime (MCR) installed on your computer to be able running the extractor. As MathWorks (MatLab developing company) restricts making MCR publicly available, I cannot make it downloadable freely from this page. However, it is allowed to provide MCR directly upon request. So, if you need MCR and cannot find it by yourself, just drop me a mail (see my contacts below) and I'll provide you with MCR package downloading link (the whole package is about 250 Mb).
Here are the instructions on how to run the watermark extractor. In the Windows command prompt run:
wm_stereo_dec.exe <watermarked_file.wav> <private_key.mat>
For example, to extract watermark data from the file gazebo_wm1_transcoded.wav (from the table above) just run:
wm_stereo_dec.exe jarre_out_wm1.wav jarre_key_(4_0.75).mat
After some time of processing, the utility will show you found watermark and will report on any errors occured during extraction (if any).
Scheme II - "strict watermarking" scheme
It is very simple and at the same time extremely effective scheme exploting the signals time-shifting idea in its most pronounced and direct way. The scheme operates with audio signals having any number of channels (including mono signals). Source (unmodified) audio signal is required to extract the watermark.
Briefly on fundamental features and restrictions of this scheme:
- Requires source (unmodified) audio data to extract watermarking data;
- Can be applied to audio data with any number of channels (including mono);
- Provides imperceptible watermarking;
- Extremely robust to all kinds of transcoding including sample rate conversion, re-quantization, transcoding using extremely low bit rate psychoacoustic subband coders like MP3, WMA, Ogg Vorbis, etc.;
- Robust to transducing via air and effects processing;
- Scalable watermarking data capacity (depends on required reliability and embedding parameters);
- Potentially can be applied not only to original PCM data, but also to already compressed files like MP3 with no need to decode them to waveform and then back to MP3.
Target applications:
- audio recording and production studios, on-line music stores, broadcasting companies having audio databases;
- can be used as an authentication mechanism for all types of audio reproduction software and hardware products - the scheme can be used to "mark" uniquely each particular exemplar of audio product to allow its further identification.
Below is my demonstration implementation of this watermarking technique. This implementation is written on MatLab and is available in a form of stand-alone executable application for Windows. It is relatively slow (compared to how fast it could be being implemented on C/C++) as it was developed for demonstration, research and evaluation purposes only. On the other hand, it is fully functional watermarking solution, capable to demonstrate abilities of the technique.
Several watermarked signals together with watermark extractor (decoder) are available for direct downloading on the page below. Watermark embedding utility (encoder) is provided upon request.
A number of key notes on this particular implementation of the scheme:
- Operates with stereo and mono audio files (WAV PCM 44.1 KHz, 16 bit);
- Source (unmodified) data is needed to decode watermark;
- Watermarking is performed in frequency subbands providing imperceptible watermarking by the human ear
- Source and watermarked audio signals should be accurately synchronized in time to allow correct watermark extraction; the decoder checks whether the signals are synchronized or not and tries to synchronize them automatically.
- Demonstrates very high robustness to spectrum and wave manipulations, sample rate conversion, re-quantization, effects processing, extremely low bitrate transcoding and even transducing via air.
I do not provide the encoder here, it is available upon request. Instead, I provide several audio examples of source and watermarked audio signals, and the watermark extractor to play with (see below).
Source audio signals (WAV PCM, 44.1 Khz, 16 bit, stereo, 50 seconds long each, 4410 Kbytes)
bach
bachms
gazebo
jarre
We will encode these four audio signals with the following watermarks:
WM1 = 0x01234567 (4 bytes)
WM2 = 0xABCDEF123456 (6 bytes)
WM3 = 0x0987654321AB (6 bytes)
The table below contains a list of downloadable files obtained by embedding each of the above watermarks (WM1, ..., WM4) into each of the four test signals. Private keys for watermark extraction are also provided.
Source file |
Used watermark | Resulting watermarked file |
Watermark payload size |
| gazebo.wav jarre.wav bach.wav brahms.wav |
WM1=0x01234567 |
gazebo_strict_out_wm1.wav |
4 bytes 6 bytes 6 bytes 4 bytes 6 bytes 6 bytes 4 bytes 6 bytes 6 bytes 4 bytes 6 bytes 6 bytes |
As it was mentioned before, this watermarking scheme also shows very strong robustness even to transducing via air. To prove this fact I put several files that were obtained by recording watermarked audio tracks with regular PC microphone placed near headphones (here is a photo of the recording setup). I was not patient enough to record all the above watermarked examples, but I put three of them:
gazebo_strict_out_wm1_transduced.wav
jarre_strict_out_wm1_transduced.wav
In addition to the presented audio examples, I provide the watermark extractor (decoder) which you can use to extract watermarking data from the example audio files presented above, and check out the results. Of course, you can also examine the robustness of this watermarking scheme by applying different audio transforms and effects, and trying to decode the watermark from the transformed files.
You can download the watermark extractor here (win32/64 application):
wm_strict_decoder_x86.zip (Windows x86 executable, 6 Mb, ZIP)
wm_strict_decoder_x64.zip (Windows x64 executable, 6 Mb, ZIP)
Watermark embedding utility (encoder) is available upon request.
Note: You will have to have MatLAB Component Runtime (MCR) installed on your computer to be able running the extractor. As MathWorks (MatLab developing company) restricts making MCR publicly available, I cannot make it downloadable freely from this page. However, it is allowed to provide MCR directly upon request. So, if you need MCR and cannot find it by yourself, just drop me a mail (see my contacts below) and I'll provide you with MCR package downloading link (the whole package is about 250 Mb).
Here are the instructions on how to run the extractor. In the Windows command prompt run:
wm_strict_dec.exe <source.wav> <watermarked.wav> <watermark data length>
For example, to extract watermark data from the file jarre_strict_out_wm1.wav (from the table above) just run:
wm_strict_dec.exe jarre.wav jarre_strict_out_wm1.wav 4
After some time of processing, the utility will show you found watermark and will report on any errors occured during extraction (if any).
An independent researcher Branislav Fratrik from Slovak University of Technology has examined my watermarking technique as a part of his diploma, has compared it to a competitive spread-spectrum method and here is his conclusion (please note that this conclusion relates to earlier implementation of technique which used Reed-Solomon encoding):
Both tested methods are very interesting and should be of use. Time-shifting implementation (both schemes) is very robust even if carried watermark is damaged by period swapping (my method). Only in about 50% of tests was the watermark detectable in case of spread-spectrum method. So its robustness against period swapping is not so good, almost insufficient.
/May 18th, 2008/
If you're interested in licensing of the above technology, or on any other matter, please do not hesitate to contact me by the following e-mail address:
mailbox /at/ radzishevsky /dot/ info.
Additional contact information can be found at my home site: http://radzishevsky.info
Links
Digital watermarking
MathWorks MatLAB
Ogg Vorbis
Lame MP3 Encoder