I have opus packets which is merged from 2 frames.
Size of packet can be great than 255 byte (max segment size in ogg page)
I can split this packet into two packets with single frame and save each in separated segment.
But... Can we simple split packet into parts\chunks 255 bytes + last part less then 255
ang put it in sequence segments into ogg page?
It doesn't work to me (maybe incorrect code). But I am interesting how to handle it properly? We really need to repack doubled packets to separated packet? or not?
Sorry for my pure english.
No need to repacketize. I forgot about special case when length of packet is 255, we need add 0 to segment table
Related
There is a problem I currently cannot resolve and need help.
I have transmitter and receiver devices. Transmitter needs to transmit random byte sequence (with possible repetitions) or unknown length. The sequence can be transformed (encoded) before transmission if we need.
Receiver device receives the sequence bytewise. It strongly requires to not have repetitioned bytes in incoming sequence. Every new byte must be different from the previous received one.
The question is how to encode the input byte sequence on transmitter side to avoid repetitions in receiver incoming byte sequence?
All bytes of incoming sequence should be uniquely decoded on receiver side.
I've heard about scramblers. As I understand some of they can output byte sequence without repetitions. But is there some simpler way?
Ignoring the first byte, the restriction that you can't have repeat bytes means that every byte represents a 1-in-255 choice, not a one-in-256. That means you can send slightly less than 8 bits per byte (7.994353 bits)
Hence, coding theory tells us that you need to transform your 256-symbol input stream into a 255-symbol stream. You then encode this 255-symbol stream by remembering the previous byte you sent out. If the symbol you want to send is lower than the previous byte you send, you send it unmodified, else you send it +1.
The decoding algorithm is the reverse. If you receive a byte that higher than the previously received byte, subtract one.
As a simple example, consider sending 254 254 254. The first one can be sent straight away (first symbol), the second will be sent as 255 (+1) and the next one will be 254 again. Thus the receiver sees 254 255 254. The only byte that's higher than the preceding byte is 255, so subtract one from that to recover the initial sequence 254 254 254.
This coding is the most efficient possible, we just have the minor challenge of mapping a random byte stream (256 symbols) to a 255-symbol stream. Remember, in this 255-symbol code, duplications are allowed. That's the reason why we invented it.
One easy but inefficient hack is to replace 254 with 254 0 and 255 with 254 1. The downside is that this effectively uses 15.998 bits for those two inputs. One difficult but perfectly space-efficient hack is to consider the whole input as a base-256 number, and convert it to base-255.
What exactly you choose probably depends on your input.
You can send data as padded group of 8 bytes, with 7 bytes of data and one special byte which has 7 bits to set 0 or 1 depends if it requires corresponding data byte to be modified (xor with FF or something) to make them different than previous. Last bit is used to make this byte different than last data one.
original data:
0x00 0x00 0x00 0x00 0x00 0x00 0x00
packet:
0x00 0xFF 0x00 0xFF 0x00 0xFF 0x00 0b01010100 <- last bit would be flipped if previous data byte is the same as this byte
Note: if you do not have buffer for 8 bytes you can send padding byte first keep it in register, left/right shift and process next byte based on bit value.
A simple approach is to just use the top bit for uniqueness, and the lower 7 bits to carry data:
0xxxxxxx 1xyyyyyy 0yyzzzzz 1zzz....
This encodes every 7 bytes of input as 8 bytes on the connection. On the sender, you have one bit of state to toggle between 0 and 1, and a 0-6 counter for the variable bitshifts. On the receiver, you don't even need to decode the top bit, so you just have the 0-6 counter to reverse the bit shifts. On both sides, you need to keep part of one byte as well, so you need about 2 bytes of state for this. Still, not too bad and certainly doable in an FPGA or about a dozen ARM instructions.
Guys what do you think about idea to use some soft byte scrambler?
I mean some simple algorithm than will transform original byte stream into pseudo-random sequence without repetitions?
Is it possible to avoid repetitions reliably this way?
Just would like to know it as additional possible solution...
I am making a simple png image from scratch. I have had the scanlines data for it. Now I want to make it into zlib stream without being compressed. How can I do that? I have read the "ZLIB Compressed Data Format Specification version 3.3" at "https://www.ietf.org/rfc/rfc1950.txt" but still not understanding. Could someone give me a hint about setting the bytes in zlib stream?
Thanks in advance!
As mentioned in RFC1950, the details of the compression algorithm are described in another castle RFC: DEFLATE Compressed Data Format Specification version 1.3 (RFC1951).
There we find
3.2.3. Details of block format
Each block of compressed data begins with 3 header bits
containing the following data:
first bit BFINAL
next 2 bits BTYPE
Note that the header bits do not necessarily begin on a byte
boundary, since a block does not necessarily occupy an integral
number of bytes.
BFINAL is set if and only if this is the last block of the data
set.
BTYPE specifies how the data are compressed, as follows:
00 - no compression
[... a few other types]
which is the one you wanted. These 2 bits BTYPE, in combination with the last-block marker BFINAL, is all you need to write "uncompressed" zlib-compatible data:
3.2.4. Non-compressed blocks (BTYPE=00)
Any bits of input up to the next byte boundary are ignored.
The rest of the block consists of the following information:
0 1 2 3 4...
+---+---+---+---+================================+
| LEN | NLEN |... LEN bytes of literal data...|
+---+---+---+---+================================+
LEN is the number of data bytes in the block. NLEN is the
one's complement of LEN.
So, the pseudo-algorithm is:
set the initial 2 bytes to 78 9c ("default compression").
for every block of 32768 or less bytesᵃ
if it's the last block, write 01, else write 00
... write [block length] [COMP(block length)]ᵇ
... write the immediate data
repeat until all data is written.
Don't forget to add the Adler-32 checksum at the end of the compressed data, in big-endian order, after 'compressing' it this way. The Adler-32 checksum is to verify the uncompressed, original data. In the case of PNG images, that data has already been processed by its PNG filters and has row filter bytes appended – and that is "the" data that gets compressed by this FLATE-compatible algorithm.
ᵃ This is a value that happened to be convenient for me at the time; it ought to be safe to write blocks as large as 65535 bytes (just don't try to cross that line).
ᵇ Both as words with the low byte first, then high byte. It is briefly mentioned in the introduction.
I have an application, that reads opus packets from a file. The file confirms opus packets in ogg format. My application sends each opus packet every 20 millisecond (it is configurable).
For 20 millisec, it sends packets of sizes ranging from 200 bytes to 400 bytes, say average size is 300 bytes.
Sending 300 bytes for 20millsec, is it right or its too much of data. How can I calculate for 20millisec how much data (in bytes) I can send to remote.
Can somebody help me to understand how to calculate number of bytes I need to send to remote party per 20millisec.
300 bytes/packet × 8 bits/byte / 20 ms/packet = 120 kbit/s
That is enough for good quality stereo music. Depending on the quality that you need, or if you are only sending mono or voice, you could potentially reduce the bitrate of the encoder. However if you are reading from an Ogg Opus file then the packets are already encoded, so it is too late to reduce the bitrate of the encoder unless you decode the packets and re-encode them at a lower bitrate.
i want help in OPUS DTX
1.how OPUS DTX is working.
2.how opus encoder and decoder works for dtx
3.Is there any bit representation for DTX in OPUS
From rfc When DTX is enabled, only one frame is encoded
every 400 milliseconds it didnt works for me with opus_demo binary.
please help with OPUS DTX.
When using Opus over a network, using a protocol such as RTP where the packets are timestamped, DTX may be enabled if you want to reduce the packets sent during periods where there is no voice activity. A packet would still be sent about every 400 ms, updating background noise. Using the packet timestamps the receiver can determine the duration of any gaps and fill them in with the background noise to keep it sounding natural.
Enable DTX in the encoder using opus_encoder_ctl(enc, OPUS_SET_DTX(1));, or with the -dtx option on opus_demo. Then, simply do not send any packets produced by the encoder with a length of 2 bytes or less. (These "DTX packets" are just zero-length frames, with a normal 1- or 2-byte packet header (TOC), and do not contain any audio data.) Packets with a length larger than 2 bytes should be sent as usual.
The receiver should use normal packet loss concealment to handle missing packets, in the same manner as it would handle packet loss. In particular it can call opus_decode() with data = NULL, len = 0, and frame_size equal to the size of the missing frame(s), and the decoder will generate audio data to conceal the missing frame(s).
Let's say you're transferring a file of arbitrary length in chunks over TCP/IP:
looping...
read(buffer, LENGTH)
send(mysocket, buffer, LENGTH, flags)
My question is, what would be optimal value of LENGTH? Or does it not matter at all? I've seen everything from 256 bytes to 8192 bytes being used.
Depends what you mean by optimal. For optimal usage of the bandwidth, you want to maximize the packet size so send at least the network packet size (which on Ethernet is usually about 1500 bytes). If you are reading from disk 4096 or 8192 bytes would be a good value.
If your buffer size translates into packet size, then shorter buffers are better -- less to retransmit in event of a packet error.
ATM took this to the extreme with a 54-byte packet.
But depending upon your library, it might be doing some buffering of its own and setting its packet size independantly. YMMV.
If you are sending large amounts of data over a high latency connection, you can get better throughput with a larger send buffer. Here is a good explanation:
http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html