I am facing this issue after using the mac mailing service.
able to read attachments if I use outlook or gmail.
piece of code where I am reading the attachment from mail.
if (contentType.contains("multipart"))
{
Multipart multiPart = (Multipart) msg.getContent();
int partCount = multiPart.getCount();
for (int j = 0; j < partCount; j++)
{
BodyPart part = multiPart.getBodyPart(j);
if (Part.ATTACHMENT.equalsIgnoreCase(part.getDisposition())) // getting null here
{
attachmentProcessing = true;
InputStream is;
is = part.getInputStream();
f = new File(filePath + part.getFileName());
fileArray.add(f);
FileOutputStream fos = new FileOutputStream(f);
byte[] buf = new byte[4096];
int bytesRead;
while((bytesRead = is.read(buf))!=-1) {
fos.write(buf, 0, bytesRead);
}
fos.close();
}
}
}
part details for mac mailing service:
text/plain; charset=us-ascii
part.getDisposition() = null
multipart/mixed; boundary="Apple-Mail=_C50D7E2D-27A1-4449-BAA3-5DC9D1E522AF"
and for others
multipart/mixed; boundary="----=_NextPart_000_0033_01D28DC6.C9D0C8B0"
text/plain; name=HashTagList.txt; charset=us-ascii
part.getDisposition() = attachment
Disposition is optional; not all messages with "attachments" will set the disposition. The JavaMail FAQ has more information.
Also, rather than checking for the string "multipart" in the content type, you should use the isMimeType method and check for "multipart/mixed".
See the msgshow.java sample program for more examples and details.
Related
sorry for my English)))
I have been trying to solve my problem for a long time, but unfortunately so far without success ((((
I have a POST request of a certain type, I need to change it and send it to another address, and the original request should go further to its address. I'm trying to work with the exchange. More to the point, I have a request like this :
https://api.binance.com/api/v3/order?symbol=CELRBTC&orderId=73902412&recvWindow=55000×tamp=1635266807744&signature=556b9c4121eb819d96a440b54661181e75ab2324a9d3b5fe0a73dd793626cb96
I need to send this request after the original one
https://fapi/binance.com/fapi/v1/order?symbol=KEEPUSDT&side=BUY&type=MARKET&quantity=100&timeInForce=GTC&recvWindow=5000×tamp=1591702613943&signature= 3c661234138461fcc7a7d8746c6558c9842d4e10870d2ecbedf7777cad694af
In addition, the public API is passed in the header
Connection: Keep-Alive
Content-Type: application/x-www-form-urlencoded
Accept: application/json, text/plain; q=0.9, text/html;q=0.8,
Accept-Charset: UTF-8, *;q=0.8
Accept-Encoding: gzip, deflate
X-MBX-APIKEY: YIGXAtXbQg0790CnIvKzo3oIQPmEPwiySjdQj28h0H3g87D2rwcun0kRWvh4m9J6
Host: api.binance.com
I tried to do all this work in the onBeforeRequest method
static function OnBeforeRequest(oSession: Session) {
if (oSession.RequestMethod == "POST" && oSession.uriContains("order") ) {
oSession.utilDecodeResponse()
var strBody=oSession.GetRequestBodyAsString();
// timestamp, time ms
var timestampSTART= oSession.url.IndexOf("timestamp=") + 10;
var timestampEND= oSession.url.IndexOf("&", timestampSTART);
var timestamp = oSession.url.Substring(timestampSTART, 13);
// SYMBOL
var symbolStart = oSession.url.IndexOf("symbol=")+7;
var symbolend = oSession.url.IndexOf("BTC", symbolStart)+3;
var symbol = oSession.url.Substring(symbolStart, symbolend-symbolStart);
// Signature (timestamp+SecretAPIKey=Hmac Sha256) theoretically, it can be taken from the original request, but it is more reliable to make your own
var signStart = oSession.url.IndexOf("signature=")+10;
var sign = oSession.url.Substring(signStart);
//PRICE
var PriceStart = oSession.url.IndexOf("price=")+6;
var PriceEND = oSession.url.IndexOf("&", PriceStart);
var priceStr = oSession.url.Substring(PriceStart, PriceEND-PriceStart);
var price = parseFloat(priceStr);
// Quantity
var quantity = 50/ price*63000;
var apiBIN = "https://fapi.binance.com/fapi/v1/order?" ;
// var result = apiBIN+"symbol="+symbol+"&side=BUY&type=MARKET&quantity="+ quantity+"&timeInForce=GTC&recvWindow=5000×tamp="+timestamp+"&signature="+sign;
// oSession.utilSetRequestBody(result)
// FiddlerApplication.oProxy.SendRequest(oSession.RequestHeaders, oSession.requestBodyBytes, null);
have selected the necessary parameters from the request, but I do not understand how to send it in any way, and I also lose the header with the API key.
Separately, I want to note the "Signature" parameter, which is created using the Hmac Sha256 algorithm from the secret API key and time, which I also can't figure out how to describe it in the code.
I would be very grateful for any help, with the possibility of some payment for substantial help.
I used C# to solve the problem. I didn't know Fiddler could choose the language. Technically, I have achieved my goal, but currently finance is writing an incorrect request. I will deal with this separately. Special thanks to Robert for his help.
public static void OnBeforeResponse(Session oSession)
{
if (oSession.HTTPMethodIs("POST") && oSession.uriContains("order"))
{
String strBody = oSession.GetRequestBodyAsString();
//Price
int PriceStart = strBody.IndexOf("price=")+6;
int PriceEND = strBody.IndexOf("&", PriceStart);
string priceStr = strBody.Substring(PriceStart, PriceEND-PriceStart);
float priceF = float.Parse(priceStr, System.Globalization.CultureInfo.InvariantCulture);
// SYMBOL
int symbolStart = strBody.IndexOf("symbol=")+7;
int symbolend = strBody.IndexOf("BTC", symbolStart);
string symbol = strBody.Substring(symbolStart, symbolend-symbolStart);
// Quantity
int quantStart = strBody.IndexOf("quantity=")+9;
int quantend = strBody.IndexOf("&price", quantStart);
string quant = strBody.Substring(quantStart, quantend-quantStart);
float quantity = float.Parse(quant, System.Globalization.CultureInfo.InvariantCulture)*2;
// timestamp
decimal timestamp = Math.Round(Convert.ToDecimal(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalMilliseconds), 0);
//Sign
Encoding ascii = Encoding.ASCII;
string secretKey = "lfaeGkjitNwvyG2lqDueMhSAOzRFlzL73w5pKRCAvSy7YrxyTkvwKCcHBHj...";
HMACSHA256 hmac = new HMACSHA256(ascii.GetBytes(secretKey));
// string query_string_LIMIT = "symbol="+symbol+"USDT&side=BUY&type=LIMIT&timeInForce=GTC&quantity="+quantity+"&price="+priceF+"&recvWindow=5000×tamp="+timestamp+"&signature=";
string result = "symbol="+symbol+"USDT&side=BUY&type=MARKET&quantity="+ quantity+"&recvWindow=5000×tamp="+timestamp+"&signature=";
String signature = BitConverter.ToString(hmac.ComputeHash(ascii.GetBytes(result))).Replace("-","");
oSession.host="fapi.binance.com";
string resultRequest = "symbol="+symbol+"USDT&side=BUY&type=MARKET&quantity="+ quantity+"&recvWindow=5000×tamp="+timestamp+"&signature="+signature;
byte[] resulByte = System.Text.Encoding.ASCII.GetBytes(resultRequest);
//oSession.utilReplaceInRequest("api/v3/order","fapi/v1/order?"+resultFin);
oSession.url = oSession.url.Replace("api/v3/order", "fapi/v1/order?"+resultRequest);
FiddlerApplication.oProxy.SendRequest (oSession.RequestHeaders, resulByte, null);
}
Is it possible to retrieve the body content of email, email header details and email attachments in Single step using Apache Nifi.
If so Please help me how to achieve this.
It is not possible in a single step unless you write your own processor or script (using ExecuteScript or InvokeScriptedProcessor). However it is possible in a single flow with something like the following:
ConsumePOP3 -> ExtractEmailHeaders -> ExtractEmailAttachments -> ...
At the end of the flow above, you will have one flow file per attachment, each flow file containing the email headers as attributes and the attachment as the content.
You can use the processor "ExecuteScript", not developing custom processor.
import email
import mimetypes
from email.parser import Parser
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from java.io import BufferedReader, InputStreamReader
from org.apache.nifi.processors.script import ExecuteScript
from org.apache.nifi.processor.io import InputStreamCallback
from org.apache.nifi.processor.io import StreamCallback
class PyInputStreamCallback(InputStreamCallback):
_text = None
def __init__(self):
pass
def getText(self) :
return self._text
def process(self, inputStream):
self._text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
flowFile = session.get()
if flowFile is not None :
reader = PyInputStreamCallback()
session.read(flowFile, reader)
msg = email.message_from_string(reader.getText())
body = ""
if msg.is_multipart():
for part in msg.walk():
ctype = part.get_content_type()
cdispo = str(part.get('Content-Disposition'))
if ctype == 'text/plain' and 'attachment' not in cdispo:
body = part.get_payload(decode=True) # decode
break
else:
body = msg.get_payload(decode=True)
flowFile = session.putAttribute(flowFile, 'msgbody', body.decode('utf-8', 'ignore'))
session.transfer(flowFile, ExecuteScript.REL_SUCCESS)
Screenshot
The gist above posted by #Thomas mostly worked for me.
However, there were cases where the multipart mail was multi-level, i.e. one of the parts was also multipart. To address that, you could use this version of the getTextFromMessage method instead:
private String getTextFromMessage(Part part) throws MessagingException, IOException {
String result = null;
if (part.isMimeType("text/plain")){
// If the part is a plaintext message, just return the content as a String
Object content = part.getContent();
if (content instanceof String) {
result = (String)content;
}
} else if (part.isMimeType("multipart/*")) {
// If the part is a multi-part message, iterate over the sub-parts
MimeMultipart mimeMultipart = (MimeMultipart)part.getContent();
int count = mimeMultipart.getCount();
for (int i = 0; i < count; i ++){
final BodyPart bodyPart = mimeMultipart.getBodyPart(i);
// Inserted from here...
final String text = getTextFromMessage( bodyPart );
if (text != null) {
result = text;
break;
}
}
}
// Just to be safe...
result = (result != null) ? result : "";
return result;
}
Note that it is possible that there is not even a "html/plain" part at all in the email, so you may want to think about how to handle that case as well. For instance, you could loop over the parts again to extract the "text/html" part (if it exists) and deal with that, and so on.
I'm using EWS with JScript (I know, not recommended, but I have no choice!)...
I can connect to my Exchange server, log in, retrieve messages, and identify a message with a file attachment. Now I want to save that attachment locally.
I have:
for (x = 0; x < myMessages.Items.Count - 1; x++)
{
thisitem = myMessages.Items.get_Item(x);
if (thisitem.HasAttachments)
{
attachment = thisitem.Attachments.get_Item(0)
attachment.Load("c:\\temp.xls");
But this produces an error:
mscorlib
System.ArgumentException: Object of type "System.String" cannot be converted to type "System.IO.Stream"
....
I know the attachment is an XLS.... and it's something to do with the overloading of the Load method... But beyond that I'm lost.
Help?!
For anyone needing to do something like this, in the end I found this worked (this is Jscript in TestComplete - amend as you need to!):
if (thisitem.HasAttachments)
{
for (y = 0; y < thisitem.Attachments.Count; y++)
{
var attachment = thisitem.Attachments.get_Item(y);
if (attachment.get_IsInline() == false)
{
if (attachment.name.OleValue.indexOf(attachsuffix) == attachment.name.OleValue.length - attachsuffix.length)
{
attachment.Load();
aqFileSystem.CreateFolder(targetfolder)
var filename = targetfolder + attachment.Name.OleValue
var fos = dotNET.System_IO.BinaryWriter.zctor(dotNET.System_IO.File.Open(filename, dotNET.System_IO.FileMode.Create))
fos.write_3(attachment.content)
fos.Close();
Log.Message("Saved: " + filename);
}
}
}
}
I want to replace a particular text in PDF document. I am currently using itextSharp library to play with PDF documents.
I had extracted the bytes from pdfdocument and then replaced that byte and then write the document again with the bytes but it is not working. In the below example I am trying to replace string 1234 with 5678
Any advise on how to perform this would be helpful.
PdfReader reader = new PdfReader(opf.FileNames[i]);
byte[] pdfbytes = reader.GetPageContent(1);
PdfString oldstring = new PdfString("1234");
PdfString newstring = new PdfString("5678");
byte[] byte1022 = oldstring.GetOriginalBytes();
byte[] byte1067 = newstring.GetOriginalBytes();
int position = 0;
for (int j = 0; j <pdfbytes.Length ; j++)
{
if (pdfbytes[j] == byte1022[0])
{
if (pdfbytes[j+1] == byte1022[1])
{
if (pdfbytes[j+2] == byte1022[2])
{
if (pdfbytes[j+3] == byte1022[3])
{
position = j;
break;
}
}
}
}
}
pdfbytes[position] = byte1067[0];
pdfbytes[position + 1] = byte1067[1];
pdfbytes[position + 2] = byte1067[2];
pdfbytes[position + 3] = byte1067[3];
File.WriteAllBytes(opf.FileNames[i].Replace(".pdf","j.pdf"), pdfbytes);
What makes you think 1234 is part of the page's content stream and not of a form XObject? Your code is never going to work in general if you don't parse all the resources of a page.
Also: I see GetPageContent(), but I don't see you using SetPageContent() anywhere. How are the changes ever going to be stored in the PdfReader object?
Moreover, I don't see you using PdfStamper to write the altered PdfReader contents to a file.
Finally: I'm to shy to quote the words of Leonard Rosenthol, Adobe's PDF Architect, but ask him, and he'll tell you personally that you shouldn't do what you're trying to do. PDF is NOT a format for editing.Read the intro of chapter 6 of the book I wrote on iText: http://www.manning.com/lowagie2/samplechapter6.pdf
I am trying to read a xml file from the web and parse it out using XDocument. It normally works fine but sometimes it gives me this error for day:
**' ', hexadecimal value 0x1F, is an invalid character. Line 1, position 1**
I have tried some solutions from Google but they aren't working for VS 2010 Express Windows Phone 7.
There is a solution which replace the 0x1F character to string.empty but my code return a stream which doesn't have replace method.
s = s.Replace(Convert.ToString((byte)0x1F), string.Empty);
Here is my code:
void webClient_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
using (var reader = new StreamReader(e.Result))
{
int[] counter = { 1 };
string s = reader.ReadToEnd();
Stream str = e.Result;
// s = s.Replace(Convert.ToString((byte)0x1F), string.Empty);
// byte[] str = Convert.FromBase64String(s);
// Stream memStream = new MemoryStream(str);
str.Position = 0;
XDocument xdoc = XDocument.Load(str);
var data = from query in xdoc.Descendants("user")
select new mobion
{
index = counter[0]++,
avlink = (string)query.Element("user_info").Element("avlink"),
nickname = (string)query.Element("user_info").Element("nickname"),
track = (string)query.Element("track"),
artist = (string)query.Element("artist"),
};
listBox.ItemsSource = data;
}
}
XML file:
http://music.mobion.vn/api/v1/music/userstop?devid=
0x1f is a Windows control character. It is not valid XML. Your best bet is to replace it.
Instead of using reader.ReadToEnd() (which by the way - for a large file - can use up a lot of memory.. though you can definitely use it) why not try something like:
string input;
while ((input = sr.ReadLine()) != null)
{
string = string + input.Replace((char)(0x1F), ' ');
}
you can re-convert into a stream if you'd like, to then use as you please.
byte[] byteArray = Encoding.ASCII.GetBytes( input );
MemoryStream stream = new MemoryStream( byteArray );
Or else you could keep doing readToEnd() and then clean that string of illegal characters, and convert back to a stream.
Here's a good resource for cleaning illegal characters in your xml - chances are, youll have others as well...
https://seattlesoftware.wordpress.com/tag/hexadecimal-value-0x-is-an-invalid-character/
What could be happening is that the content is compressed in which case you need to decompress it.
With HttpHandler you can do this the following way:
var client = new HttpClient(new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.GZip
| DecompressionMethods.Deflate
});
With the "old" WebClient you have to derive your own class to achieve the similar effect:
class MyWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}
Above taken from here
To use the two you would do something like this:
HttpClient
using (var client = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }))
{
using (var stream = client.GetStreamAsync(url))
{
using (var sr = new StreamReader(stream.Result))
{
using (var reader = XmlReader.Create(sr))
{
var feed = System.ServiceModel.Syndication.SyndicationFeed.Load(reader);
foreach (var item in feed.Items)
{
Console.WriteLine(item.Title.Text);
}
}
}
}
}
WebClient
using (var stream = new MyWebClient().OpenRead("http://myrss.url"))
{
using (var sr = new StreamReader(stream))
{
using (var reader = XmlReader.Create(sr))
{
var feed = System.ServiceModel.Syndication.SyndicationFeed.Load(reader);
foreach (var item in feed.Items)
{
Console.WriteLine(item.Title.Text);
}
}
}
}
This way you also recieve the benefit of not having to .ReadToEnd() since you are working with the stream instead.
Consider using System.Web.HttpUtility.HtmlDecode if you're decoding content read from the web.
If you are having issues replacing the character
For me there were some issues if you try to replace using the string instead of the char. I suggest trying some testing values using both to see what they turn up. Also how you reference it has some effect.
var a = x.IndexOf('\u001f'); // 513
var b = x.IndexOf(Convert.ToString((byte)0x1F)); // -1
x = x.Replace(Convert.ToChar((byte)0x1F), ' '); // Works
x = x.Replace(Convert.ToString((byte)0x1F), " "); // Fails
I blagged this
I had the same issue and found that the problem was a embedded in the xml.
The solution was:
s = s.Replace(" ", " ")
I'd guess it's probably an encoding issue but without seeing the XML I can't say for sure.
In terms of your plan to simply replace the character but not being able to, because you have a stream rather than a text, simply read the stream into a string and then remove the characters you don't want.
Works for me.........
string.Replace(Chr(31), "")
I used XmlSerializer to parse XML and faced the same exception.
The problem is that the XML string contains HTML codes of invalid characters
This method removes all invalid HTML codes from string (based on this thread - https://forums.asp.net/t/1483793.aspx?Need+a+method+that+removes+illegal+XML+characters+from+a+String):
public static string RemoveInvalidXmlSubstrs(string xmlStr)
{
string pattern = "&#((\\d+)|(x\\S+));";
Regex regex = new Regex(pattern, RegexOptions.IgnoreCase);
if (regex.IsMatch(xmlStr))
{
xmlStr = regex.Replace(xmlStr, new MatchEvaluator(m =>
{
string s = m.Value;
string unicodeNumStr = s.Substring(2, s.Length - 3);
int unicodeNum = unicodeNumStr.StartsWith("x") ?
Convert.ToInt32(unicodeNumStr.Substring(1), 16)
: Convert.ToInt32(unicodeNumStr);
//according to https://www.w3.org/TR/xml/#charsets
if ((unicodeNum == 0x9 || unicodeNum == 0xA || unicodeNum == 0xD) ||
((unicodeNum >= 0x20) && (unicodeNum <= 0xD7FF)) ||
((unicodeNum >= 0xE000) && (unicodeNum <= 0xFFFD)) ||
((unicodeNum >= 0x10000) && (unicodeNum <= 0x10FFFF)))
{
return s;
}
else
{
return String.Empty;
}
})
);
}
return xmlStr;
}
Nobody can answer if you don't show relevant info - I mean the Xml content.
As a general advice I would put a breakpoint after ReadToEnd() call. Now you can do a couple of things:
Reveal Xml content to this forum.
Test it using VS Xml visualizer.
Copy-paste the string into a txt file and investigate it offline.