I was wondering if it's possible to transfer arbitrary binary data through messaging apps such as Telegram. I guess the question is if binary data can be transferred through text messages. I read somewhere that this is possible if Base64 binary-to-text encoding is used. Telegram is a platform which is not censored in the country I'm living in. So, if I can relqy binary data through telegram, it can be used to bypass censorship. Does telegram support Base64 encoding? What are your thoughts on this?
Well, I'm almost certain that the Base64 encoding can be used to carry binary data on telegram. However, there are limitations on the rate of messages sent on Telegram. Therefore, the idea to use Telegram as a proxy is not achievable since a significantly high number of messages needs to be sent.
Related
I wanted to know which kind of protocol apache beam uses to read and write to cloud storage. Is it HTTPS or Binary(Blob).I tried to google it but I did not find. I know gsutil command uses HTTPS protocol.
You are mixing 2 things: Transport layer and data encoding.
Does Google use HTTP transport? YES, for all the API. HTTPS or gRPC (HTTP/2) are commonly use.
Does Google use binary encoding to speed up the transfert? As said before, transport can be HTTPS or gRPC. HTTPS is commonly use for REST API, and transport JSON text format. Of course, you can exchange files in binary format (GZIP for example to compress and speed up the transfert). gRPC is a binary protocol. You don't exchange JSON, but binary representation of the data that you want to exchange. And thereby, the file transfert is also in binary mode.
Now, what use Beam? As often, the Google libraries use gRPC behind the scene, and thus, the encoding is binary. If you perform yourselves REST API call, with JSON, and HTTP will be use for that; but file content is, when it can (depends on your request accept content header), transferred in binary.
EDIT 1
For BEAM, I had a look to the source code. You have here, for example, the creation of GoogleCloudStorageImpl object.
If you have a look to this full class name: import com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl;. Ok, let's see the hadoop package!!
The Javadoc is clear: JSON API is used. to confirm that, I went to the source code, and YES, the JSON format is used for the API communication.
BUT, keep in mind that is the API communication, the metadata around the file content. The file content should be sent in binary format (plain text of b64 encoding should be strange).
I am currently working on an endpoint for the Postmark service. Postmark sends received E-Mails as JSON and put the attachments as Base64 into an "Attachment"-Array.
My question is now if it is possible to use akka streams to extract the Attachments while receiving them and "do my things" with it?"
In details: I wanna slice the Attachments into chunks, hash them while they come in to store the chunks in my database.
The rest of the incoming JSON can be processed (without) the attachments afterwards.
The reason I wanna do this, is to use as little memory as possible becuase the attachment sizes can go up to 100MB and the hashing of such big files can use a lot of processing power at once.
I hope to save memory and to much cpu load if I process the attachments in chunks.
If you know a better way to achieve this I am open for any suggestion! :)
Based on a what I found on the internet, MIME (Multipurpose Internet Mail Extensions, now Internet Media Type (?)) is a way to describe file types (a header used by several protocols).
So, MIME itself is not a protocol, rather an extension used by other protocols, right ?
This means that the extension is used at the application layer by the applications with no protocol doing anything other than carrying the MIME header.
So, if I send a mail with a mp3 attachment, SMTP/other application layer protocol recognizes that this is an mp3 attachment or it is the duty of the application solely to recognize the file? In that sense, MIME cannot be called as an extension to SMTP but rather a feature to be used by applications.
If SMTP does not recognize that this is a different kind of file, how will it properly store it at the mail server ? (e.g. a MPEG video file needs a particular format to be stored, how will mail server store it without giving it any special treatment ? )
Sorry if my questions sound a bit vague but I want to get an idea of how different protocols (especially, SMTP) use MIME.
Thanks for your help.
RFC 822 email was originally purely plain-text, 7-bit US-ASCII. MIME specifies a facility for encapsulating other media types in email containers. It does not specify any changes to SMTP (although e.g. the 8BITMIME ESMTP extension is useful for simplifying transport of MIME messages). Thus, it is an extension of an existing protocol, not a distinct protocol in its own right. This is also demonstrated by the fact that other protocols -- notably, HTTP -- have incorporated (parts of) MIME for tagging of content types and encodings.
An Internet Media Type is only one aspect of what MIME used to codify; the mechanisms for specifying character sets and encodings are still defined in MIME proper.
Traditionally, the mail server simply stores the bare RFC822 message in its message store; it is the responsibility of the mail client to parse and possibly manipulate any MIME structure in the body for display and interaction. (The fact that RFC 822 has been superseded by 2282 and then 5322 has not fundamentally changed the actual mail message format.)
Some servers deviate from this model; for example, Microsoft Exchange seems to parse all incoming messages in order to borg them into its internal format, somewhat to the detriment of its interoperability with standard tools, and the sanity of those few of us who require reliable, felicitous access to our actual email.
The SMTP protocol itself knows nothing about the MIME format, but the SMTP server itself has to at least implement basic rfc0822 support in order to ad the Received headers, however, it does not need to implement MIME.
How does the server save the file to disk? The same way it received it from the client over the TCP/IP stream. It just saves the raw bytes sent (with the addition of the addition of a Received header I mentioned).
In other words, you are way over-thinking this. The SMTP server doesn't have to know anything about mp3 file attachments or anything else because the MIME format (it's not a protocol) is just a way to serialize the mp3 data in a message.
I am currently working on xmppframework, Requirements are to transfer the file between two iPhones. I searched for XEPs and found 0065 and 0096
XEP-0065 says:
XMPP is designed for sending relatively small chunks of XML between
network entities and is not designed for sending binary data. However,
sometimes it is desirable to send binary data to another entity that
one has discovered on the XMPP network (e.g., to send a file).
Therefore it is valuable to have a generic protocol for streaming
binary data between any two entities on an XMPP network. The main
application for such a bytestreaming technology is file transfer as
specified in SI File Transfer [1] and Jingle File Transfer [2].
However, other applications are possible, which is why it is important
to develop a generic protocol rather than one that is specialized for
a particular application such as file transfer.
Please see the line in bold, its confusing me if file transfer XEPs are SI File Transfer(0096) and Jingle File Transfer(0234), then what is the purpose of this 0065 XEP? why people on net referring sep-0065 for file transfer?
In XMPP there are different protocols (XEPS) for file transfer. Jingle, Bytestreams, OOB, IBB...
The purpose of XEP-0096 is stream initiation. So its build on top of the other file transfer protocols to enable seamless file transfers.
So its used to agree on one of the above file transfer protocols between 2 clients for a transfer, and also for finding a fallback method if this fails for any reason.
Alex
XEP-0065 is for proxied file transfers: you will need such a proxy in your infrastructure, unless you use a public one.
XEP-0096 is much more complex, I wouldn't recommend that for a start, although I would recommend it if you later extensively use large binary transfers/exchanges, as Jingle is used for VoIP at least.
Can anyone provide any direction or links on how to use the adaptive bitrate feature that DSS says it supports? According to the release notes for v6.0.3:
3GPP Release 6 bit rate adaptation support
I assume that this lets you include multiple video streams in the 3gp file with varying bitrates, and DSS will automatically serve the best stream based on the current bandwidth. At least that's what I hope it does.
I guess I'm not sure what format DSS is expecting to receive the file. I tried just adding several streams to a 3gp file which resulted in Quicktime unable to play it, and VLC opening up a different window for each stream in the file.
Any direction would be much appreciated.
Adaptive streaming used in DSS 6.x uses a dropped frame approach to reduce overall bandwidth rather than dynamic on the fly bitrate adjustments. The result of this can be unpredictable. The DSS drops the frames, and does not need the video encoded in any special way for it to work.