I have been reading https://en.wikipedia.org/wiki/Line_code, but it is not obvious to me why 4b5b and 8b10b are called line code.
What is the fundamental difference between line code and more sophisticated hamming code?
What is the fundamental difference between line code and more sophisticated hamming code?
They're both serving entirely different purposes.
A line code enables clock recovery on a single, serial bit stream and provides bit-level synchronization. It may also provide byte-level synchronization. On differential electrical cabling, the line code (often in combination with a scrambler) also needs to remove DC bias.
A hamming code adds redundant bits for forward error correction.
The fundamental difference is the purpose of the code.
A line code is used to condition the data on a transmission line. Generally to balance the number of 0 and 1 bits sent to balance the current (what both 4b5b and 8b10b codes so)
A hamming code is used for error correction -- to detect transmission errors and possibly correct for them.
Related
Lately I have been reading about Manchester encoding and I think I'm beginning to understand most of it now, but still I have got some whys that need addressing. Mainly 3 for the moment:
1) Most articles on Internet when introducing Manchester coding start by telling how bad NRZI really was and one of the disadvantages that gets mentioned is that synchronization becomes a problem when lengthy 1's or 0's get sent. Why is that a problem, since most places where NRZI is used have got separate clock and data lines. As long as the clock signal is there why should that ever be a problem?
2) Also, is Manchester supposed to work on a fixed frequency? Or can it work like I2C where clock frequency can be variable?
3) The good thing that gets mentioned about Manchester encoding is that it does not require separate clock line and that clock is embedded in the data and can be recovered by the receiver. Frequent transitions in Manchester help in synchronization and that the transitions happen in the middle and so clock can be recovered from transition. But my question is, if there are repeated 1's or 0's transition can happen in the middle and in the end as well (see attached waveform pic, look at the transitions when sending 111). So when a receiver sees a transition how does it figure out whether it is in the middle or at the end?
If I'm talking rubbish I would love to be corrected.
regarding your third question: I'm also brushing up on manchester and it appears that to recover a clock you need a differential signal:
Reference: "Data Communications, Computer Networks and Open Systems" by Fred Halsall, page 104, figure 3.8
For the 3 question,
Whenever a signal is transmitted, initially a few redundant bits which contain info about clock are sent.
For example, 1111, now the receiver knows the real data will arrive next, and through those redundant bits clock signal is extracted as well as the “notification “ that a signal is going to come.
As for question 1, NRZ scheme can send lengthy 1’s and lengthy 0’s.... but here the problem is actually with lengthy 1’s, if you could check sending lengthy 1’s with some modulation scheme and a dipole antenna, you could observe that the power of carrier signal will start decaying exponentially.
And the other reason would be the power needed to send that many lengthy 1’s, which is not favourable!
For question 2, yes it is possible to use it variable clock frequency but the condition is you should send redundant bits before you could change the clock frequency so that the receiver understands that the clock is changed from this point onwards.
Hope it’s clear now ;)
From my understanding, DVB-T packets go through two FEC systems, that are, Viterbi, with a data loss up to 50%, and RS, with a data loss up to 10%. Those are called external and internal coding.
I can't understand the need for the second RS coding (in that case, MPEG-TS packets 188 bytes long are added an additional 20 bytes).
More specifically, what happens to packets that are corrupted, say, 55%? Are 50% of the errors fixed by the Viterbi decoder and the remaining 5% from the RS?
Sorry for my dumbness.
The abilities and targets of Viterbi / RS differ considerably: Viterbi coding is done next to baseband/analog level, where each bit has a high probability of being corrupted. This is combated with a scheme, where not all combinations of e.g. '00000' through '11111' are possible, but where every other or 1/3 or 2/3 bits are correction bits calculated from the history of some N previous bits transferred.
This causes a comparably high expansion of data with the possibility of correcting typically one half of individual bit errors. One has to notice that the bit errors can occur for the correction bits as well...
This kind of bit error correction can mitigate errors mostly on AWGN channels and somewhat on Rayleigh fading (simulation model for signal fading due to moving vehicle with multi-path propagation, i.e. same signal coming from multiple paths).
Because the "window" of the Viterbi encoder is small, and when there's a burst error over the complete window (e.g. 7 bits), the encoder is not able to correct any errors. Thus a secondary coder is needed: Reed Solomon (in DVB or CD) coder works with codewords of size 8 bits, i.e. when a single bit in the codeword is corrupted, the complete codeword needs to be fixed.
The idea thus is, that the outer coder can reduce sporadic single bit errors to a manageable level, leaving basically burst errors (long period of unreceived signal) to the inner coding.
I am in the process of coding a simple Genetic Algorithm (GA). There are probably countless areas where I have unnecessarily used a for loop. I would like some tips on how to be more MATLAB efficient as well as an answer to my question. As far as I can tell I have succeeded but I am not sure. The area which this code defines is single-point crossover
Here is what I have tried...
crossPoints=randi([1 24],popSize/2,1);
for popNo=2:2:popSize
isolate=chromoParent(popNo-1:popNo,crossPoints(popNo/2,1)+1:end);
isolate([1 2],:)=isolate([2 1],:);
chromoParent(popNo-1:popNo,crossPoints(popNo/2,1)+1:end)=isolate;
end
chromoChild=chromoParent;
where, 'crossPoints' is the point at which single point crossover
between two binary encoded chromosomes is required.
'popSize' is the size of the population, required by my code to
be an even number
'isolate' defines the sections of 2 rows which are required to be swapped
with each other
'chromoParent' is the initial population which is required to be
changed by single-point crossover
'chromoChild' is the resulting population
Both 'chromoParent' and 'chromoChild' are represented by an array of
size, popSize x 25 binary characters
Can you spot an error in the way I am thinking about this problem? What's the most efficient way (in computational time) to achieve the same thing? It would help if you could be as broad as possible so that I could begin applying the principles I learn here to the rest of my code.
Thank you.
Your code looks fine. If you want, you can reduce the instructions in the loop to a single line by some very simple indexing:
chromoParent( popNo-1:popNo, crossPoints(popNo/2,1)+1:end) = ...
chromoParent(popNo:-1:popNo-1,crossPoints(popNo/2,1)+1:end);
This may be marginally faster, but as with any optimization, you should profile it first (My guess is that these line contribute very little to the overall CPU time).
Yamaha InfoSound and ShopKick application use technologies that allow to transfer data using ultrasound. That is playing an inaudible signal (>18kHz) that can be picked up by modern mobile phones (iOS, Android).
What is the approach used in such technologies? What kind of modulation they use?
I see several problems with this approach. First, 18kHz is not inaudible. Many people cannot hear it, especially as they age, but I know I certainly can (I do regular hearing tests, work-related). Also, most phones have different low-pass filters on their A/D converters, and many devices, especially older Android ones (I've personally seen that happen), filter everything below 16 kHz or so. Your app therefore is not guaranteed to work on any hardware. The iPhone should probably be able to do it.
In terms of modulation, it could be anything really, but I would definitely rule out AM. Sound has next to zero robustness when it comes to volume. If I were to implement something like that, I would go with FSK. I would think that PSK would fail due to acoustic reflections and such. The difficulty is that you're working with non-robust energy transfer within a very narrow bandwidth. I certainly do not doubt that it can be achieved, but I don't see something like this proving reliable. Just IMHO, that is.
Update: Now that i think about it, a plain on-off would work with a single tone if you're not transferring any data, just some short signals.
Can't say for Yamaha InfoSound and ShopKick, but what we used in our project was a variation of frequency modulation: the frequency of the carrier is modulated by a digital binary signal, where 0 and 1 correspond to 17 kHz and 18 kHz respectively. As for demodulator, we tried heterodyne. More details you could find here: http://rnd.azoft.com/mobile-app-transering-data-using-ultrasound/
There's nothing special in being ultrasound, the principle is the same as data transmission through a modem, so any digital modulation is -in principle- feasible. You only have a specific frequency band (above 18khz) and some practical requisites (the medium is very unreliable, I guess) that suggest to use a simple-robust scheme with low-bit rate.
I don't know how they do it but this is how I do it:
If it is a string then make sure it's not a long one (the longer the higher is the error probability ). Lets assume we're working with the vital part of the ASCII code, namely up to character number 127, then all you need is 7 bits per character. Transform this character into bits and modulate those bits using QFSK (there are several modulations to choose from, frequency shift based ones have turned out to be the most robust I've tried from the conventional ones... I've created my own modulation scheme for this use case). Select the carrier frequencies as 18.5,19,19.5, and 20 kHz (if you want to be mathematically strict in your design, select frequency values that assure you both orthogonality and phase continuity at symbol transitions, if you can't, a good workaround to avoid abrupt symbols transitions is to multiply your symbols by a window of the same size, eg. a Gaussian or Bartlet ). In my experience you can move this values in the range from 17.5 to 20.5 kHz (if you go lower it will start to bother people using your app, if you go higher the average type microphone frequency response will attenuate your transmission and induce unwanted errors).
On the receiver side implement a correlation or matched filter receiver (an FFT receiver works as well, specially a zero padded one but it might be a little bit slower, I wouldn't recommend Goertzel because frequency shift due to Doppler effect or speaker-microphone non-linearities could affect your reception). Once you have received the bit stream make characters with them and you will recover your message
If you face too many broadcasting errors, try selecting a higher amount of samples per symbol or band-pass filtering each frequency value before giving them to the demodulator, using an error correction code such as BCH or Reed Solomon is sometimes the only way to assure an error free communication.
One topic everybody always forgets to talk about is synchronization (to know on the receiver side when the transmission has begun), you have to be creative here and make a lot a tests with a lot of phones before you can derive an actual detection threshold that works on all, notice that this might also be distance dependent
If you are unfamiliar with these subjects I would recommend a couple of great books:
Digital Modulation Techniques from Fuqin Xiong
DIGITAL COMMUNICATIONS Fundamentals and Applications from BERNARD SKLAR
Digital Communications from John G. Proakis
You might have luck with a library I created for sound-based modems, libquiet. It gives you a handful of profiles to work from, including a slow "Ultrasonic whisper" profile with spectral content above 19kHz. The library is written in C but would require some work to interface with iOS.
This is more of a computer science / information theory question than a straightforward programming one, so if anyone knows of a better site to post this, please let me know.
Let's say I have an N-bit piece of data that will be sent redundantly in M messages, where at least M-1 of those messages will be received successfully. I am interested in different ways of encoding the N-bit piece of data in fewer bits per message. (this is similar to RAID but at a much smaller level, where N = 8 or 16 or 32)
Example: suppose N = 16 and M = 4. Then I could use the following algorithm:
1st and 3rd message: send "0" + bits 0-7
2nd and 4th message: send "1" + bits 8-15
If I can guarantee that 3 messages of the 4 will get through, then at least one message from each group will get through. Thus I can make this work with 9 bits or less, there's probably a way to do this with fewer total bits but I'm not sure how.
Are there some simple encoding/decoding algorithms to do this kind of thing? Does this problem have a name? (if I know what it's called, I can google it!)
note: in my particular case, the messages either arrive correctly or do not arrive at all (no messages arrive with errors).
(edit: moved 2nd part to a separate question)
(Incomplete answer follows. I may add more later.)
The term you may be interested in is channel coding: adding redundancy to a source in order to make it robust during transmission over a noisy channel. In information theory, the complementary problem to channel coding is source coding: reducing the redundancy in a source to represent it using fewer bits. (The combination of these two problems is called joint source-channel coding.)
Your first question asks to find a channel code. The simple example you give is similar to a repetition code, i.e., you send the same message more than twice (usually an odd number of times), and then the message which is received most often is accepted as the original message.
This code is inefficient. To use standard notation, let k = number of bits in original message, and n = number of bits in the transmitted message. For your example, k = 16 and n = 36. A measure of coding efficiency is k/n, where higher means more efficient. In your case, k/n = 0.44. This is low.
The repetition code is a simple kind of block code, i.e., redundancy is added to each block of k bits to create a codeword of n bits. So are the Hamming and Reed-Solomon codes as others mentioned. Hamming codes are relatively easy to understand with some basic linear algebra.
These should be enough terms for you to search on your own. Good luck.
I'm not sure if I understood all the details of your question correctly, but your problem is definitely aboud designing some kind of error correcting code. This is a vast area of computer science and thick tomes have been written about it. Start with wikipedia and see if you can get any simple schemes (like Hamming or Reed-Solomon codes) to work in your case.
If you want to deal not only with symbol corruption, but also deletion of symbols, you should look at erasure codes, this is definitely a more difficult task but good methods exist in many cases.
EDIT: This material from hackersdelight.org seems a nice introduction.
See erasure codes.
You're looking for a packet erasure code. There are only two useful packet erasure codes that are not totally encumbered by patents, and there's only one open-source library to implement those. Find it here: http://planete-bcast.inrialpes.fr/rubrique.php3?id_rubrique=5
Here's a trivially simple scheme that's almost twice as efficient as your example.
You chopped the message into blocks of (N/M)*2 bits. Instead, chop it into N/(M-1)-bit blocks. (Round it up if necessary.) The first block, src[0], encodes as itself: enc[0]=src[0]. The same for the last block: enc[M-1]=src[M-1]. Each of the other blocks gets XORed with its left neighbor: enc[i]=src[i-1]^src[i].
Prefix each encoded block with a log(M)-bit sequence number, essentially as you did, so the receiver can tell which was dropped. (If you can be sure that whichever blocks arrive will arrive in order, then a 1-bit sequence number will do. Just alternate 0 and 1.)
To decode, successively XOR from the left and the right until you hit the dropped block. E.g. src[1] == enc[0]^enc[1]. (Dropping one of the endpoint blocks isn't a special case -- e.g. if the first block is dropped, the scan from the right recovers it, and the scan from the left is of length 0.)