Zend FW response object and image data - adding linefeed? - zend-framework

I have run into a problem using php in Zend Framweork to dynamically scale images for return as mime-type image/jpeg.
The problem manifests itself as firefox reporting 'cannot display image because it contains errors'. This is the same problem reported in: return dynamic image zf2
To replicate the problem, I removed any file IO and copied code from a similar stack overflow example verbatim (in a zend FW action controller):
$resp = $this->getRespose();
$myImage = imagecreate(200,200);
$myGray = imagecolorallocate($myImage, 204, 204, 204);
$myBlack = imagecolorallocate($myImage, 0, 0, 0);
imageline($myImage, 15, 35, 120, 60, $myBlack);
ob_start();
imagejpeg($myImage);
$img_string = ob_get_contents();
$scaledSize = ob_get_length();
ob_end_clean();
imagedestroy($myImage);
$resp->setContent($img_string);
$resp->getHeaders()->addHeaders(array(
'Content-Type' => $mediaObj->getMimeType(),
'Content-Transfer-Encoding' => 'binary'));
$resp->setStatusCode(Response::STATUS_CODE_200);
return $resp;
When I use wget to capture the jpeg response, I notice a '0A' as the first byte of the output rather than the field separator 'FF'. There is no such '0A' in the data captured in the buffer, nor in the response's content member. Attempting to open the wget output with GIMP fails, unless I remove the 0A. I am guessing that Zend FW is using the line-feed as a field separator for the response fields vs. the content, but I'm not sure if that is the problem, or if it is, how to fix it.
My response fields look OK:
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Sat, 02 Jun 2018 23:30:09 GMT
Server: Apache/2.4.7 (Ubuntu) OpenSSL/1.0.1f
Set-Cookie: PHPSESSID=nsgk1o5au7ls4p5g6mr9kegoeg; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Content-Transfer-Encoding: binary
Content-Length: 1887
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: image/jpeg
Here is the dump of the first few bytes of the wget with the jpeg stream that fails:
00000000 0a ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 |.......JFIF.....|
00000010 60 00 60 00 00 ff fe 00 3e 43 52 45 41 54 4f 52 |......>CREATOR|
Any idea where the '0A' is coming from? I am running zend framework 2.5.1, PHP 7.2.2

Thank you Tim Fountain.
I did finally find the offending file buried in some doctrine entities I had created. Sure enough, a stray "?>" with an empty line.
Much appreciated

Related

Troubleshooting connectivity from a pod in kubernetes

I have created a pod and service, called node-port.
root#hello-client:/# nslookup node-port
Server: 10.100.0.10
Address: 10.100.0.10#53
Name: node-port.default.svc.cluster.local
Address: 10.100.183.19
I can enter inside a pod and see the resolution happening.
However, the TCP connection is not happening from a node.
root#hello-client:/# curl --trace-ascii - http://node-port.default.svc.cluster.local:3050
== Info: Trying 10.100.183.19:3050...
What are the likely factors contributing to failure?
What are some suggestions to troubleshoot this?
On a working node/cluster. I expect this to work like this.
/ # curl --trace-ascii - node-port:3050
== Info: Trying 10.100.13.83:3050...
== Info: Connected to node-port (10.100.13.83) port 3050 (#0)
=> Send header, 78 bytes (0x4e)
0000: GET / HTTP/1.1
0010: Host: node-port:3050
0026: User-Agent: curl/7.83.1
003f: Accept: */*
004c:
== Info: Mark bundle as not supporting multiuse
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 38 bytes (0x26)
0000: Server: Werkzeug/2.2.2 Python/3.8.13
<= Recv header, 37 bytes (0x25)
0000: Date: Fri, 26 Aug 2022 04:34:48 GMT
<= Recv header, 32 bytes (0x20)
0000: Content-Type: application/json
<= Recv header, 20 bytes (0x14)
0000: Content-Length: 25
<= Recv header, 19 bytes (0x13)
0000: Connection: close
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 25 bytes (0x19)
0000: {. "hello": "world".}.
{
"hello": "world"
}
== Info: Closing connection 0
/ #

Which block represents a WARC-Block-Digest?

At Line 09 below there is this line: WARC-Block-Digest: sha1:CLODKYDXCHPVOJMJWHJVT3EJJDKI2RTQ
Line 01: WARC/1.0
Line 02: WARC-Type: request
Line 03: WARC-Target-URI: https://climate.nasa.gov/vital-signs/carbon-dioxide/
Line 04: Content-Type: application/http;msgtype=request
Line 05: WARC-Date: 2018-11-03T17:20:02Z
Line 06: WARC-Record-ID: <urn:uuid:e44bc1ea-61a1-4200-b94f-60042456f638>
Line 07: WARC-IP-Address: 54.230.195.16
Line 08: WARC-Warcinfo-ID: <urn:uuid:6d14bf1d-0ef7-4f03-9de2-e578d105d3cb>
Line 09: WARC-Block-Digest: sha1:CLODKYDXCHPVOJMJWHJVT3EJJDKI2RTQ
Line 10: Content-Length: 141
Line 11:
Line 12: GET /vital-signs/carbon-dioxide/ HTTP/1.1
Line 13: User-Agent: Wget/1.15 (linux-gnu)
Line 14: Accept: */*
Line 15: Host: climate.nasa.gov
Line 16: Connection: Keep-Alive
WARC's specs say that The WARC-Block-Digest is an optional parameter indicating the algorithm name and calculated value of a digest applied to the full block of the record.
I've been trying to figure out what full block of the record refers to. Is it line 11 to 16? Or Line 12 to 16? Or Line 1 to 16 (without line 9)? I've tried hashing those possibilities but can't get the sha1 (base 32) value above.
A WARC record of a HTTP GET requests has three parts (cf. the WARC spec):
the WARC header
the HTTP request header
the payload which is empty (note: a POST requests would include a non-empty payload)
The payload digest of the record is the base32-encoded SHA-1 of the empty string. A proof using Linux command-line tools:
$> echo -n "" | openssl dgst -binary -sha1 | base32
3I42H3S6NNFQ2MSVX7XZKYAYSCX5QBYJ
A WARC record has the form:
warc-record = header CRLF
block CRLF CRLF
(see WARC spec: record model)
The "full" block should include everything up to the trailing \r\n\r\n. This means lines 11 to 17. Note: also the HTTP GET request ends with \r\n\r\n (a trailing blank line):
$> cat request
GET /vital-signs/carbon-dioxide/ HTTP/1.1
User-Agent: Wget/1.15 (linux-gnu)
Accept: */*
Host: climate.nasa.gov
Connection: Keep-Alive
$> tail -n2 request | hexdump -C
00000000 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 4b 65 65 70 |Connection: Keep|
00000010 2d 41 6c 69 76 65 0d 0a 0d 0a |-Alive....|
0000001a
$> cat request | openssl dgst -binary -sha1 | base32
CLODKYDXCHPVOJMJWHJVT3EJJDKI2RTQ

Responding with a stream sometimes result in "Connection reset by peer" error

In our application we have routes that are streaming JSON documents. Here is an example:
/** GET api/1/tenant/(tenantId)/ads/ */
def getAllAdsByOwner(advertiserId: AdvertiserId): Route =
get {
httpRequiredSession { username =>
getAllTenantAds(username, advertiserId) { (adSource: Source[AdView, Any]) =>
complete(adSource)
}
}
}
Most of the time it works as expected, but sometimes, especially when there are many simultaneous requests, the server starts resetting connection just after the headers have been sent.
I tested with a script that requests this route with curl in a loop and aborting if the request failed. It was running for about 2 minutes before stopping. Trace when request fails is the following:
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 54 bytes (0x36)
0000: Access-Control-Allow-Origin: https://<...>
<= Recv header, 135 bytes (0x87)
0000: Access-Control-Expose-Headers: Content-Type, Authorization, Refr
0040: esh-Token, Set-Authorization, Set-Refresh-Token, asset-content-l
0080: ength
<= Recv header, 40 bytes (0x28)
0000: Access-Control-Allow-Credentials: true
<= Recv header, 24 bytes (0x18)
0000: Content-Encoding: gzip
<= Recv header, 23 bytes (0x17)
0000: X-Frame-Options: DENY
<= Recv header, 33 bytes (0x21)
0000: X-Content-Type-Options: nosniff
<= Recv header, 26 bytes (0x1a)
0000: Content-Security-Policy: .
<= Recv header, 20 bytes (0x14)
0000: default-src 'self';.
<= Recv header, 63 bytes (0x3f)
0000: style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;.
<= Recv header, 59 bytes (0x3b)
0000: font-src 'self' 'unsafe-inline' https://fonts.gstatic.com;.
<= Recv header, 99 bytes (0x63)
0000: script-src 'self' 'unsafe-inline' 'unsafe-eval' https://*.google
0040: apis.com https://maps.gstatic.com;.
<= Recv header, 69 bytes (0x45)
0000: img-src 'self' data: https://*.googleapis.com https://*.gstatic.
0040: com;.
<= Recv header, 8 bytes (0x8)
0000:
<= Recv header, 26 bytes (0x1a)
0000: Server: akka-http/10.1.3
<= Recv header, 37 bytes (0x25)
0000: Date: Wed, 27 Jun 2018 15:20:24 GMT
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 32 bytes (0x20)
0000: Content-Type: application/json
<= Recv header, 2 bytes (0x2)
0000:
== Info: Recv failure: Connection reset by peer
== Info: stopped the pause stream!
== Info: Closing connection 0
curl: (56) Recv failure: Connection reset by peer
The same request inspected in Wireshark:
screen shot
Reading logs didn't give any hint about probable source of the problem. Response logged as successful:
[27-06-2018 19:44:52.837][INFO] access: 'GET /api/1/tenant/ca764a91-8616-409c-8f08-c64a40d3fc07/ads' 200 596ms
Versions of used software:
Scala: 2.11.11
akka: 2.5.13
akka-http: 10.1.3
Configuration:
akka.conf
akka-http-core.conf
I tried increasing akka.http.host-connection-pool.max-connections to 128 but it didn't help. Maybe someone has an idea if this is a bug in akka-http or configuration problem?
If there is no I/O on your open connection for the idle-timeout, Akka will close the connection which often appears as a "connection reset by peer" error. Try increasing the akka.http.server.idle-timeout value.
Because your akka.http.server.request-timeout value is the same as akka.http.server.idle-timeout, it is a race condition between which timeout will occur first when there is no I/O. Sometimes, you will see a 503; other times, you will experience a connection reset error.

MimeMultipart count is zero when an email is read using JavaMail

My application sends an email to an Exchange mail server, mail server is configured with a third party application where it routes email to agent and agent replies to that email. Application reads agent reply from the mailbox which is used to send the email.
Email sending code is below;
Message mimeMessage = new MimeMessage(session);
mimeMessage.setFrom(new InternetAddress(from));
mimeMessage.addRecipient(Message.RecipientType.TO, new InternetAddress(to));
mimeMessage.setSubject(subject);
mimeMessage.setContent(emailText,"text/plain");
mimeMessage.setReplyTo(replyToAddress);
Transport.send(mimeMessage);
This works perfectly. When agent reply is received, Application read it as;
if (message.isMimeType("multipart/MIXED")) {
logger.info("Email MIME Type is: multipart/MIXED");
MimeMultipart multipart =(MimeMultipart)message.getContent();
logger.info("Content type = "+multipart.getContentType());
int count = multipart.getCount();
}
The content type is "multipart/mixed" but the count is 0 means there are no parts in this emails.
I need to set System property,
System.setProperty("mail.mime.multipart.allowempty", "true");
if it is not set, multipart.getCount() throws "missingBoundryException".
Why it is so ?
I can see that the agent's reply is not empty.
The email was sent with content type as text/plain, why reply type is multipart/mixed?
Is this due to any invalid formatting of email by third party application, what is the workaround?
Below is the snap of agent reply.
Below is the raw MIME content,
Received: from sociaminer.host (192.168.1.29) by thirdpartHost
(192.168.1.53) with Microsoft SMTP Server (TLS) id 14.1.218.12; Thu, 19 Jan
2017 17:06:26 +0500
To: hafiz <hafiz#bla.bla>
Message-ID: <hassan.MESSAGEID#bla.bla>
In-Reply-To: <CF72F94#bla.bla>
References: <CF72F945A#bla.bla>
Subject: Re: 1122+50
Content-Type: multipart/mixed;
boundary="----=_Part_127_14151461.1484827604583"
From: <reply#bla.bla>
Return-Path: reply#bla.bla
Date: Thu, 19 Jan 2017 17:06:26 +0500
X-MS-Exchange-Organization-AuthSource: bla.bla
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 06
X-Originating-IP: [SocialMinerIP]
MIME-Version: 1.0
------=_Part_127_14151461.1484827604583
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">Reply to 50<br>
<blockquote><hr>
<b>From:</b> hafiz <hafiz#bla.bla><br><b>Sent:</b> Thursday, January 19, 2017 5:05 PM<br><b>To:</b> testing2 <testing2#bla.bla><br><b>Subject:</b> 1122+50<br>
<html dir="ltr">
<head>
<style type="text/css" id="owaParaStyle"></style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Testing 50</div>
</body>
</html>
</blockquote>
------=_Part_127_14151461.1484827604583--
JavaMail debug output looks like below,
DEBUG: setDebug: JavaMail version 1.4.7
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: disable AUTH=PLAIN
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: trying to connect to host "Echange IP", port 143, isSSL false
* OK The Microsoft Exchange IMAP4 service is ready.
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 LOGINDISABLED STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: protocolConnect login, host=192.168.1.53, user=hafiz#bla.bla, password=<non-null>
A1 STARTTLS
A1 OK Begin TLS negotiation now.
A2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A2 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTHENTICATE NTLM command trace suppressed
DEBUG NTLM: type 1 message: 4E 54 4C 4D 53 53 50 00 01 00 00 00 03 A2 00 00 00 00 00 00 23 00 00 00 03 00 03 00 20 00 00 00 31 39 32
DEBUG NTLM: type 3 message: 4E 54 4C 4D 53 53 50 00 03 00 00 00 18 00 18 00 68 00 00 00 18 00 18 00 80 00 00 00 00 00 00 00 40 00 00 00 22 00 22 00 40 00 00 00 06 00 06 00 62 00 00 00 00 00 00 00 98 00 00 00 01 82 00 00 68 00 61 00 66 00 69 00 7A 00 40 00 65 00 66 00 6C 00 61 00 62 00 2E 00 6C 00 6F 00 63 00 61 00 6C 00 31 00 39 00 32 00 3B 5E 2B 86 67 49 E3 01 C9 9E F2 CA ED 54 21 11 81 89 94 C6 EC E0 26 E3 BA DB E7 5A F4 CA 28 17 7C 0E 8A 08 18 B5 5A 4E 72 4F C5 7F 52 64 FA 76
DEBUG IMAP: AUTHENTICATE NTLM command result: A3 OK AUTHENTICATE completed.
A4 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A4 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: connection available -- size: 1
A5 SELECT INBOX
* 40 EXISTS
* 0 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 39] Is the first unseen message
* OK [UIDVALIDITY 436] UIDVALIDITY value
* OK [UIDNEXT 46] The next unique identifier value
A5 OK [READ-WRITE] SELECT completed.
A6 SEARCH UNSEEN ALL
* SEARCH 39
A6 OK SEARCH completed.
A7 SEARCH UNSEEN ALL
* SEARCH 39
A7 OK SEARCH completed.
main INFO emailToSms.EmailReader - 1 unread emails read from inbox.
A8 STORE 39 +FLAGS (\Seen)
* 39 FETCH (FLAGS (\Seen))
A8 OK STORE completed.
A9 FETCH 39 (BODY.PEEK[HEADER])
* 39 FETCH (BODY[HEADER] {851}
MIME-Version: 1.0
Received: from HOST (IP) by HOST
(192.168.1.53) with Microsoft SMTP Server (TLS) id 14.1.218.12; Thu, 19 Jan
2017 17:06:26 +0500
To: hafiz <hafiz#bla.bla>
Message-ID: <hassan.B69E3DD110000159000004A73F57FEE3.1484827604448.cisco-ccp#bla.bla>
In-Reply-To: <CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>
References: <CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>
Subject: Re: 1122+50
Content-Type: multipart/mixed;
boundary="----=_Part_127_14151461.1484827604583"
From: <testing2#bla.bla>
Return-Path: testing2#bla.bla
Date: Thu, 19 Jan 2017 17:06:26 +0500
X-MS-Exchange-Organization-AuthSource: Expert.bla.bla
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 06
X-Originating-IP: [IP]
)
A9 OK FETCH completed.
A10 FETCH 39 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 39 FETCH (ENVELOPE ("Thu, 19 Jan 2017 17:06:26 +0500" "Re: 1122+50" ((NIL NIL "testing2" "bla.bla")) NIL NIL (("hafiz" NIL "hafiz" "bla.bla")) NIL NIL "<CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>" "<hassan.B69E3DD110000159000004A73F57FEE3.1484827604448.cisco-ccp#bla.bla>") INTERNALDATE "19-Jan-2017 17:06:26 +0500" RFC822.SIZE 1250)
A10 OK FETCH completed.
A11 FETCH 39 (BODYSTRUCTURE)
* 39 FETCH (BODYSTRUCTURE ("multipart" "mixed" ("boundary" "----=_Part_127_14151461.1484827604583") NIL NIL 7BIT 0 NIL NIL NIL NIL))
A11 OK FETCH completed.
DEBUG IMAP: IMAPProtocol noop
A12 NOOP
A12 OK NOOP completed.
This is a bug in Microsoft Exchange. Report this bug to Microsoft and upgrade to a newer version or newer service pack if possible in case they've already fixed it.
Exchange is returning the BODYSTRUCTURE information for the message as if it were a single part message when in fact it is a multipart message. This is a violation of the IMAP protocol spec.
You can use the workaround in the JavaMail FAQ.
Also, you might want to upgrade to a newer version of JavaMail - 1.4.7 is pretty old, the current version is 1.5.6.

Sending a trap with Perl's Net::SNMP

I'm trying to send a trap as part of a larger Perl script. I've copied the trapsending code to another file, and am running it by itself. The code seems to think the trap sends successfully, yet I'm not seeing the trap on either machine that I have a trap listener running on.
Here's the code:
#! /usr/local/bin/perl
use strict;
use warnings;
use Net::SNMP;
#messy hardcoding
my $snmp_target = '192.168.129.50';
#my $snmp_target = '10.200.6.105'; # Server running trap listener
my $enterprise = '1.3.6.1.4.1.27002.1';
my ($sess, $err) = Net::SNMP->session(
-hostname => $snmp_target,
-version => 1, #trap() requires v1
);
if (!defined $sess) {
print "Error connecting to target ". $snmp_target . ": ". $err;
next;
}
my #vars = qw();
my $varcounter = 1;
push (#vars, $enterprise . '.' . $varcounter);
push (#vars, OCTET_STRING);
push (#vars, "Test string");
my $result = $sess->trap(
-varbindlist => \#vars,
-enterprise => $enterprise,
-specifictrap => 1,
);
if (! $result)
{
print "An error occurred sending the trap: " . $sess->error();
}
EDIT: Added $sess->debug(255) call, here's the output:
debug: [440] Net::SNMP::Dispatcher::_event_insert(): created new head and tail [ARRAY(0x1af1fea8)]
debug: [687] Net::SNMP::Message::send(): transport address 192.168.129.50:161
debug: [2058] Net::SNMP::Message::_buffer_dump(): 70 bytes
[0000] 30 44 02 01 00 04 06 70 75 62 6C 69 63 A4 37 06 0D.....public.7.
[0016] 09 2B 06 01 04 01 81 D2 7A 01 40 04 C0 A8 81 85 .+......z.#.....
[0032] 02 01 06 02 01 01 43 01 00 30 1B 30 19 06 0A 2B ......C..0.0...+
[0048] 06 01 04 01 81 D2 7A 01 01 04 0B 54 65 73 74 20 ......z....Test
[0064] 73 74 72 69 6E 67 string
debug: [517] Net::SNMP::Dispatcher::_event_delete(): deleted [ARRAY(0x1af1fea8)], list is now empty
EDIT: Can anyone running a trap listener try this code on their machine and let me know if it works?
EDIT: Tried it from my MBP. Same result. Then noticed that the debug info says it is sending to port 161. Forced -port => 162 parameter, and it works. That leaves me with a couple of questions:
Why does the trap sender default to 161?
I get this error when I run with debug on. What does it mean?
error: [97] Net::SNMP::Transport::IPv4::UDP::agent_addr(): Failed to disconnect: Address family not supported by protocol family
Fixed by changing 'Port' setting from default 161 to 162.