Special characters not working despite UTF-8 in Response Header [closed] - encoding
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 13 days ago.
This post was edited and submitted for review 13 days ago.
Improve this question
In my AngularJS app I'm using JSON translation files, which get loaded into the app via HTTP requests. I see that HTTP requests in the network, and in the Preview I see that the content is corrupted (special characters are not shown correctly (see screenshot below).
However, in the Response Headers I see that it's encoded as UTF-8 like this:
content-type: application/json;charset=UTF-8
However, the special characters are not shown correctly:
I did another test and uploaded the file to another server and opened the URL (from the other server) as well.
The strange thing is, that on the other server the same file works perfectly and the characters are shown correctly, so it must be some server issue.
Also, when I download the file and analyze its encoding, it says UTF-8 in both cases (on the working server, and also on the non-working server).
Is there any server configuration which might prevent from delivering the file in a correct format?
The server where it's non-working is an nginx server.
The Response Headers:
accept-ranges: bytes
Connection: Keep-Alive
Content-Encoding: gzip
content-language: de-DE
content-type: application/json;charset=UTF-8
Date: Tue, 07 Feb 2023 09:10:44 GMT
etag: "63c9737f-eb86-gzip"
Keep-Alive: timeout=15, max=100
last-modified: Thu, 19 Jan 2023 16:44:47 GMT
Server: Apache
strict-transport-security: max-age=31536000
Transfer-Encoding: chunked
vary: Accept-Encoding
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
Request Headers:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Connection: keep-alive
Host: www.something.de
If-Modified-Since: Thu, 19 Jan 2023 16:44:47 GMT
If-None-Match: "63c9737f-eb86-gzip"
sec-ch-ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36
Raw contents of the file:
unsigned char ucDataBlock[344] = {
// Offset 0x00000000 to 0x00000157
0x7B, 0x0A, 0x20, 0x20, 0x22, 0x72, 0x69, 0x65, 0x73, 0x74, 0x65, 0x72,
0x2E, 0x70, 0x65, 0x72, 0x66, 0x6F, 0x72, 0x6D, 0x61, 0x6E, 0x63, 0x65,
0x50, 0x65, 0x72, 0x63, 0x65, 0x6E, 0x74, 0x2E, 0x6D, 0x65, 0x73, 0x73,
0x61, 0x67, 0x65, 0x2E, 0x74, 0x69, 0x74, 0x6C, 0x65, 0x22, 0x3A, 0x20,
0x22, 0x57, 0x61, 0x72, 0x75, 0x6D, 0x20, 0x6D, 0xC3, 0x83, 0xC2, 0xBC,
0x73, 0x73, 0x65, 0x6E, 0x20, 0x53, 0x69, 0x65, 0x20, 0x65, 0x69, 0x6E,
0x65, 0x20, 0x57, 0x65, 0x72, 0x74, 0x65, 0x6E, 0x74, 0x77, 0x69, 0x63,
0x6B, 0x6C, 0x75, 0x6E, 0x67, 0x20, 0x61, 0x75, 0x73, 0x77, 0xC3, 0x83,
0xC2, 0xA4, 0x68, 0x6C, 0x65, 0x6E, 0x3F, 0x22, 0x2C, 0x0A, 0x20, 0x20,
0x22, 0x72, 0x69, 0x65, 0x73, 0x74, 0x65, 0x72, 0x2E, 0x70, 0x65, 0x72,
0x66, 0x6F, 0x72, 0x6D, 0x61, 0x6E, 0x63, 0x65, 0x50, 0x65, 0x72, 0x63,
0x65, 0x6E, 0x74, 0x2E, 0x6D, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x2E,
0x62, 0x6F, 0x64, 0x79, 0x22, 0x3A, 0x20, 0x22, 0x44, 0x69, 0x65, 0x20,
0x6B, 0xC3, 0x83, 0xC2, 0xBC, 0x6E, 0x66, 0x74, 0x69, 0x67, 0x65, 0x20,
0x45, 0x6E, 0x74, 0x77, 0x69, 0x63, 0x6B, 0x6C, 0x75, 0x6E, 0x67, 0x20,
0x61, 0x6E, 0x20, 0x64, 0x65, 0x6E, 0x20, 0x4B, 0x61, 0x70, 0x69, 0x74,
0x61, 0x6C, 0x6D, 0xC3, 0x83, 0xC2, 0xA4, 0x72, 0x6B, 0x74, 0x65, 0x6E,
0x20, 0x6B, 0x61, 0x6E, 0x6E, 0x20, 0x6E, 0x69, 0x63, 0x68, 0x74, 0x20,
0x76, 0x6F, 0x72, 0x68, 0x65, 0x72, 0x67, 0x65, 0x73, 0x65, 0x68, 0x65,
0x6E, 0x20, 0x77, 0x65, 0x72, 0x64, 0x65, 0x6E, 0x2E, 0x20, 0x44, 0x61,
0x68, 0x65, 0x72, 0x20, 0x76, 0x65, 0x72, 0x77, 0x65, 0x6E, 0x64, 0x65,
0x6E, 0x20, 0x77, 0x69, 0x72, 0x20, 0x66, 0xC3, 0x83, 0xC2, 0xBC, 0x72,
0x20, 0x64, 0x69, 0x65, 0x20, 0x4C, 0x65, 0x69, 0x73, 0x74, 0x75, 0x6E,
0x67, 0x73, 0x64, 0x61, 0x72, 0x73, 0x74, 0x65, 0x6C, 0x6C, 0x75, 0x6E,
0x67, 0x20, 0x65, 0x69, 0x6E, 0x65, 0x20, 0x6D, 0x6F, 0x64, 0x65, 0x6C,
0x6C, 0x68, 0x61, 0x66, 0x74, 0x65, 0x20, 0x6A, 0xC3, 0x83, 0xC2, 0xA4,
0x68, 0x72, 0x6C, 0x69, 0x63, 0x68, 0x65, 0x20, 0x57, 0x65, 0x72, 0x74,
0x65, 0x6E, 0x74, 0x77, 0x69, 0x63, 0x6B, 0x6C, 0x75, 0x6E, 0x67, 0x2E,
0x22, 0x2C, 0x0A, 0x20, 0x20, 0x0A, 0x7D, 0x0A
};
Could anyone help and point me to a possible problem please?
Related
Connection between CLI and Peer/Orderer not working properly (Kubernetes setup)
I'm running a network in a Kubernetes Cluster and have a CLI, a Peer and an Orderer of the same organization each running in it's own Pod. I can do channel creation, chaincode installation, approvement and committing without problems. However, when it comes to chaincode invocation, the CLI outputs that the chaincode might not be installed, while the Peer logs a failed connection to the CLI. So here's the CLI command (update: with -o org1-orderer:30011): $ export CORE_PEER_MSPCONFIGPATH=/config/admin/msp $ peer chaincode invoke -C channel1 -n cc-abac -c '{"Args":["invoke","a","b","10"]}' -o org1-orderer:30011 --clientauth --tls --cafile /config/peer/tls-msp/tlscacerts/ca-cert.pem --keyfile /config/peer/tls-msp/keystore/key.pem --certfile /config/peer/tls-msp/signcerts /cert.pem CLI Output: 2020-07-07 16:47:20.918 UTC [msp] loadCertificateAt -> WARN 001 Failed loading ClientOU certificate at [/config/admin/msp]: [could not read file /config/admin/msp: read /config/admin/msp: is a directory] 2020-07-07 16:47:20.919 UTC [msp] loadCertificateAt -> WARN 002 Failed loading PeerOU certificate at [/config/admin/msp]: [could not read file /config/admin/msp: read /config/admin/msp: is a directory] 2020-07-07 16:47:20.919 UTC [msp] loadCertificateAt -> WARN 003 Failed loading AdminOU certificate at [/config/admin/msp]: [could not read file /config/admin/msp: read /config/admin/msp: is a directory] 2020-07-07 16:47:20.919 UTC [msp] loadCertificateAt -> WARN 004 Failed loading OrdererOU certificate at [/config/admin/msp]: [could not read file /config/admin/msp: read /config/admin/msp: is a directory] 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 005 parsed scheme: "" 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 006 scheme "" not registered, fallback to default scheme 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 007 ccResolverWrapper: sending update to cc: {[{org1-peer1:30151 <nil> 0 <nil>}] <nil> <nil>} 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 008 ClientConn switching balancer to "pick_first" 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 009 Channel switches to new LB policy "pick_first" 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 00a Subchannel Connectivity change to CONNECTING 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 00b Subchannel picks a new address "org1-peer1:30151" to connect 2020-07-07 16:47:20.928 UTC [grpc] UpdateSubConnState -> DEBU 00c pickfirstBalancer: HandleSubConnStateChange: 0xc000114450, {CONNECTING <nil>} 2020-07-07 16:47:20.928 UTC [grpc] Infof -> DEBU 00d Channel Connectivity change to CONNECTING 2020-07-07 16:47:20.935 UTC [grpc] Infof -> DEBU 00e Subchannel Connectivity change to READY 2020-07-07 16:47:20.935 UTC [grpc] UpdateSubConnState -> DEBU 00f pickfirstBalancer: HandleSubConnStateChange: 0xc000114450, {READY <nil>} 2020-07-07 16:47:20.935 UTC [grpc] Infof -> DEBU 010 Channel Connectivity change to READY 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 011 parsed scheme: "" 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 012 scheme "" not registered, fallback to default scheme 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 013 ccResolverWrapper: sending update to cc: {[{org1-peer1:30151 <nil> 0 <nil>}] <nil> <nil>} 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 014 ClientConn switching balancer to "pick_first" 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 015 Channel switches to new LB policy "pick_first" 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 016 Subchannel Connectivity change to CONNECTING 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 017 Subchannel picks a new address "org1-peer1:30151" to connect 2020-07-07 16:47:20.948 UTC [grpc] UpdateSubConnState -> DEBU 018 pickfirstBalancer: HandleSubConnStateChange: 0xc000496070, {CONNECTING <nil>} 2020-07-07 16:47:20.948 UTC [grpc] Infof -> DEBU 019 Channel Connectivity change to CONNECTING 2020-07-07 16:47:20.954 UTC [grpc] Infof -> DEBU 01a Subchannel Connectivity change to READY 2020-07-07 16:47:20.955 UTC [grpc] UpdateSubConnState -> DEBU 01b pickfirstBalancer: HandleSubConnStateChange: 0xc000496070, {READY <nil>} 2020-07-07 16:47:20.955 UTC [grpc] Infof -> DEBU 01c Channel Connectivity change to READY 2020-07-07 16:47:20.987 UTC [chaincodeCmd] InitCmdFactory -> INFO 01d Retrieved channel (channel1) orderer endpoint: org1-orderer:30011 2020-07-07 16:47:20.991 UTC [grpc] WithKeepaliveParams -> DEBU 01e Adjusting keepalive ping interval to minimum period of 10s 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 01f parsed scheme: "" 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 020 scheme "" not registered, fallback to default scheme 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 021 ccResolverWrapper: sending update to cc: {[{org1-orderer:30011 <nil> 0 <nil>}] <nil> <nil>} 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 022 ClientConn switching balancer to "pick_first" 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 023 Channel switches to new LB policy "pick_first" 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 024 Subchannel Connectivity change to CONNECTING 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 025 Subchannel picks a new address "org1-orderer:30011" to connect 2020-07-07 16:47:20.991 UTC [grpc] UpdateSubConnState -> DEBU 026 pickfirstBalancer: HandleSubConnStateChange: 0xc000205a60, {CONNECTING <nil>} 2020-07-07 16:47:20.991 UTC [grpc] Infof -> DEBU 027 Channel Connectivity change to CONNECTING 2020-07-07 16:47:21.000 UTC [grpc] Infof -> DEBU 028 Subchannel Connectivity change to READY 2020-07-07 16:47:21.000 UTC [grpc] UpdateSubConnState -> DEBU 029 pickfirstBalancer: HandleSubConnStateChange: 0xc000205a60, {READY <nil>} 2020-07-07 16:47:21.000 UTC [grpc] Infof -> DEBU 02a Channel Connectivity change to READY Error: endorsement failure during invoke. response: status:500 message:"make sure the chaincode cc-abac has been successfully defined on channel channel1 and try again: chaincode definition for 'cc-abac' exists, but chaincode is not installed" I'm sure it's installed on channel1 (the only channel in existence, except sys-channel): $ peer lifecycle chaincode queryinstalled Installed chaincodes on peer: Package ID: cc-abac:4992a37bf5c7b48f91f5062d9700a58a4129599c53d759e8282fdeffc8836c72, Label: cc-abac On the Peer's side, I get the following in the log (updated): [36m2020-07-09 06:45:55.976 UTC [gossip.discovery] periodicalSendAlive -> DEBU 194c[0m Sleeping 5s [36m2020-07-09 06:45:56.182 UTC [endorser] ProcessProposal -> DEBU 194d[0m request from 10.129.1.229:60184 [36m2020-07-09 06:45:56.182 UTC [endorser] Validate -> DEBU 194e[0m creator is valid channel=channel1 txID=a71312e4 mspID=Org1MSP [36m2020-07-09 06:45:56.182 UTC [msp.identity] Verify -> DEBU 194f[0m Verify: digest = 00000000 87 29 a0 e5 96 b8 5f 5e 9b e0 fb e5 4d 5b 86 b2 |.)...._^....M[..| 00000010 bd 43 ee 30 59 d6 a9 55 e3 e9 77 7b fd a2 47 8f |.C.0Y..U..w{..G.| [36m2020-07-09 06:45:56.182 UTC [msp.identity] Verify -> DEBU 1950[0m Verify: sig = 00000000 30 45 02 21 00 f0 6b 23 9d f6 ec f2 29 be 64 4e |0E.!..k#....).dN| 00000010 75 69 a7 05 7e 05 71 51 64 6c 52 59 83 be ea f9 |ui..~.qQdlRY....| 00000020 08 5e 07 09 f3 02 20 7a f7 b0 6c e0 bb 32 b9 0c |.^.... z..l..2..| 00000030 8c 41 be b8 ea 39 33 91 92 0b 08 9e c6 14 39 e8 |.A...93.......9.| 00000040 46 eb a5 80 7a 7d d1 |F...z}.| [36m2020-07-09 06:45:56.182 UTC [endorser] Validate -> DEBU 1951[0m signature is valid channel=channel1 txID=a71312e4 mspID=Org1MSP [36m2020-07-09 06:45:56.182 UTC [fsblkstorage] retrieveTransactionByID -> DEBU 1952[0m retrieveTransactionByID() - txId = [a71312e411a6b417a541112e2aeac73adc8d6f7fbbb3c62ffcad2348e0c91fac] [36m2020-07-09 06:45:56.182 UTC [leveldbhelper] GetIterator -> DEBU 1953[0m Getting iterator for range [[]byte{0x63, 0x68, 0x61, 0x6e, 0x6e, 0x65, 0x6c, 0x31, 0x0, 0x74, 0x1, 0x40, 0x61, 0x37, 0x31, 0x33, 0x31, 0x32, 0x65, 0x34, 0x31, 0x31, 0x61, 0x36, 0x62, 0x34, 0x31, 0x37, 0x61, 0x35, 0x34, 0x31, 0x31, 0x31, 0x32, 0x65, 0x32, 0x61, 0x65, 0x61, 0x63, 0x37, 0x33, 0x61, 0x64, 0x63, 0x38, 0x64, 0x36, 0x66, 0x37, 0x66, 0x62, 0x62, 0x62, 0x33, 0x63, 0x36, 0x32, 0x66, 0x66, 0x63, 0x61, 0x64, 0x32, 0x33, 0x34, 0x38, 0x65, 0x30, 0x63, 0x39, 0x31, 0x66, 0x61, 0x63}] - [[]byte{0x63, 0x68, 0x61, 0x6e, 0x6e, 0x65, 0x6c, 0x31, 0x0, 0x74, 0x1, 0x40, 0x61, 0x37, 0x31, 0x33, 0x31, 0x32, 0x65, 0x34, 0x31, 0x31, 0x61, 0x36, 0x62, 0x34, 0x31, 0x37, 0x61, 0x35, 0x34, 0x31, 0x31, 0x31, 0x32, 0x65, 0x32, 0x61, 0x65, 0x61, 0x63, 0x37, 0x33, 0x61, 0x64, 0x63, 0x38, 0x64, 0x36, 0x66, 0x37, 0x66, 0x62, 0x62, 0x62, 0x33, 0x63, 0x36, 0x32, 0x66, 0x66, 0x63, 0x61, 0x64, 0x32, 0x33, 0x34, 0x38, 0x65, 0x30, 0x63, 0x39, 0x31, 0x66, 0x61, 0x63, 0xff}] [36m2020-07-09 06:45:56.182 UTC [aclmgmt] CheckACL -> DEBU 1954[0m acl policy /Channel/Application/Writers found in config for resource peer/Propose [36m2020-07-09 06:45:56.182 UTC [aclmgmt] CheckACL -> DEBU 1955[0m acl check(/Channel/Application/Writers) [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1956[0m == Evaluating *policies.ImplicitMetaPolicy Policy /Channel/Application/Writers == [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1957[0m This is an implicit meta policy, it will trigger other policy evaluations, whose failures may be benign [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1958[0m == Evaluating *cauthdsl.policy Policy /Channel/Application/Org1/Writers == [36m2020-07-09 06:45:56.183 UTC [msp.identity] Verify -> DEBU 1959[0m Verify: digest = 00000000 87 29 a0 e5 96 b8 5f 5e 9b e0 fb e5 4d 5b 86 b2 |.)...._^....M[..| 00000010 bd 43 ee 30 59 d6 a9 55 e3 e9 77 7b fd a2 47 8f |.C.0Y..U..w{..G.| [36m2020-07-09 06:45:56.183 UTC [msp.identity] Verify -> DEBU 195a[0m Verify: sig = 00000000 30 45 02 21 00 f0 6b 23 9d f6 ec f2 29 be 64 4e |0E.!..k#....).dN| 00000010 75 69 a7 05 7e 05 71 51 64 6c 52 59 83 be ea f9 |ui..~.qQdlRY....| 00000020 08 5e 07 09 f3 02 20 7a f7 b0 6c e0 bb 32 b9 0c |.^.... z..l..2..| 00000030 8c 41 be b8 ea 39 33 91 92 0b 08 9e c6 14 39 e8 |.A...93.......9.| 00000040 46 eb a5 80 7a 7d d1 |F...z}.| [36m2020-07-09 06:45:56.183 UTC [policies] SignatureSetToValidIdentities -> DEBU 195b[0m signature for identity 0 validated [36m2020-07-09 06:45:56.183 UTC [cauthdsl] func1 -> DEBU 195c[0m 0xc0006210e0 gate 1594277156183221199 evaluation starts [36m2020-07-09 06:45:56.183 UTC [cauthdsl] func2 -> DEBU 195d[0m 0xc0006210e0 signed by 0 principal evaluation starts (used [false]) [36m2020-07-09 06:45:56.183 UTC [cauthdsl] func2 -> DEBU 195e[0m 0xc0006210e0 processing identity 0 - &{Org1MSP 0b33fd619da73c0915b76088b0678047f834593ea6a4f22f0772b36f3c6bd68f} [36m2020-07-09 06:45:56.183 UTC [cauthdsl] func2 -> DEBU 195f[0m 0xc0006210e0 principal evaluation succeeds for identity 0 [36m2020-07-09 06:45:56.183 UTC [cauthdsl] func1 -> DEBU 1960[0m 0xc0006210e0 gate 1594277156183221199 evaluation succeeds [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1961[0m Signature set satisfies policy /Channel/Application/Org1/Writers [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1962[0m == Done Evaluating *cauthdsl.policy Policy /Channel/Application/Org1/Writers [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1963[0m Signature set satisfies policy /Channel/Application/Writers [36m2020-07-09 06:45:56.183 UTC [policies] EvaluateSignedData -> DEBU 1964[0m == Done Evaluating *policies.ImplicitMetaPolicy Policy /Channel/Application/Writers [36m2020-07-09 06:45:56.183 UTC [lockbasedtxmgr] NewTxSimulator -> DEBU 1965[0m constructing new tx simulator [36m2020-07-09 06:45:56.183 UTC [lockbasedtxmgr] newLockBasedTxSimulator -> DEBU 1966[0m constructing new tx simulator txid = [a71312e411a6b417a541112e2aeac73adc8d6f7fbbb3c62ffcad2348e0c91fac] [36m2020-07-09 06:45:56.183 UTC [stateleveldb] GetState -> DEBU 1967[0m GetState(). ns=_lifecycle, key=namespaces/fields/cc-abac/Sequence [36m2020-07-09 06:45:56.183 UTC [lockbasedtxmgr] Done -> DEBU 1968[0m Done with transaction simulation / query execution [a71312e411a6b417a541112e2aeac73adc8d6f7fbbb3c62ffcad2348e0c91fac] [34m2020-07-09 06:45:56.183 UTC [comm.grpc.server] 1 -> INFO 1969[0m unary call completed grpc.service=protos.Endorser grpc.method=ProcessProposal grpc.peer_address=10.129.1.229:60184 grpc.peer_subject="CN=org1-peer1,OU=peer,O=Hyperledger,ST=North Carolina,C=US" grpc.code=OK grpc.call_duration=1.225382ms [36m2020-07-09 06:45:56.186 UTC [grpc] warningf -> DEBU 196a[0m transport: http2Server.HandleStreams failed to read frame: read tcp 10.130.2.65:7051->10.129.1.229:60184: read: connection reset by peer [36m2020-07-09 06:45:56.186 UTC [grpc] infof -> DEBU 196b[0m transport: loopyWriter.run returning. connection error: desc = "transport is closing" [36m2020-07-09 06:45:56.186 UTC [grpc] infof -> DEBU 196c[0m transport: loopyWriter.run returning. connection error: desc = "transport is closing" [update] The message unary call completed grpc.service=protos.Endorser grpc.method=ProcessProposal grpc.peer_address=10.129.1.229:60184 grpc.peer_subject="CN=org1-peer1,OU=peer,O=Hyperledger,ST=North Carolina,C=US" grpc.code=OK grpc.call_duration=1.225382ms indicates that the Peer considers the CLI as another Peer, doesn't it? If so, it's clear why the connection is failing. Now the question is, why the Peer thinks so? Peer: 10.130.2.65 CLI: 10.129.1.229 Kind regards
Unfortunately, all of the GRPC logs and k8s related issues appear to be a red herring. The connection is being correctly established, the term 'peer' is simply a little confusing in the GRPC logs, as GRPC always refers to 'the party on the other end of the line' as a 'peer'. This term is re-used with a different meaning in Fabric. As the logs indicate, the chaincode has been successfully approved, and defined on the channel. As the peer CLI output indicates, you have installed a chaincode with package-id cc-abac:4992a37bf5c7b48f91f5062d9700a58a4129599c53d759e8282fdeffc8836c72. But, on invoke, you are seeing the error that: chaincode definition for 'cc-abac' exists, but chaincode is not installed This means when you did your chaincode approval, you either did not specify a package-id, or, you specified an incorrect package id. If you are using a v2.2+ version of Fabric, you should be able to use the peer lifecycle queryapproved utility to see what package ID you have selected. You can re-run the peer lifecycle approveformyorg with the correct package-id (cc-abac:4992a37bf5c7b48f91f5062d9700a58a4129599c53d759e8282fdeffc8836c72) and this should correct things.
Run kube-proxy on fedora server 23 panic: runtime error: invalid memory address or nil pointer dereference
fedora : [root#host3 vagrant]# cat /etc/os-release NAME=Fedora VERSION="23 (Twenty Three)" ID=fedora VERSION_ID=23 PRETTY_NAME="Fedora 23 (Twenty Three)" ANSI_COLOR="0;34" CPE_NAME="cpe:/o:fedoraproject:fedora:23" HOME_URL="https://fedoraproject.org/" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=23 REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=23 PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy kube-proxy version: [root#host3 vagrant]# kube-proxy --version=true Kubernetes v1.1.2 Run command and error msg: [root#host3 vagrant]# kube-proxy --logtostderr=true --v=0 --master=http://host1:8080 --proxy-mode=userspace --cleanup-iptables=true panic: runtime error: invalid memory address or nil pointer dereference [signal 0xb code=0x1 addr=0xb1 pc=0x465e26] goroutine 1 [running]: k8s.io/kubernetes/cmd/kube-proxy/app.(*ProxyServer).Run(0xc2080d79d0, 0xc208046af0, 0x0, 0x5, 0x0, 0x0) /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kube-proxy/app/server.go:309 +0x56 main.main() /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kube-proxy/proxy.go:53 +0x225 goroutine 7 [chan receive]: github.com/golang/glog.(*loggingT).flushDaemon(0x12169e0) /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x78 created by github.com/golang/glog.init·1 /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x2a7 goroutine 17 [syscall, locked to thread]: runtime.goexit() /usr/src/go/src/runtime/asm_amd64.s:2232 +0x1 goroutine 15 [chan receive]: github.com/godbus/dbus.(*Conn).outWorker(0xc2080daa20) /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/godbus/dbus/conn.go:367 +0x58 created by github.com/godbus/dbus.(*Conn).Auth /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/godbus/dbus/auth.go:119 +0xea1 goroutine 12 [sleep]: k8s.io/kubernetes/pkg/util.Until(0xda66f0, 0x12a05f200, 0xc20800a7e0) /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/util.go:127 +0x98 created by k8s.io/kubernetes/pkg/util.InitLogs /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/logs.go:49 +0xab goroutine 14 [IO wait]: net.(*pollDesc).Wait(0xc2080d6920, 0x72, 0x0, 0x0) /usr/src/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc2080d6920, 0x0, 0x0) /usr/src/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).readMsg(0xc2080d68c0, 0xc2080e28c0, 0x10, 0x10, 0xc20816d220, 0x1000, 0x1000, 0xffffffffffffffff, 0x0, 0x0, ...) /usr/src/go/src/net/fd_unix.go:296 +0x54e net.(*UnixConn).ReadMsgUnix(0xc20803a0f0, 0xc2080e28c0, 0x10, 0x10, 0xc20816d220, 0x1000, 0x1000, 0x0, 0xc2080e276c, 0x4, ...) /usr/src/go/src/net/unixsock_posix.go:147 +0x167 github.com/godbus/dbus.(*oobReader).Read(0xc20816d200, 0xc2080e28c0, 0x10, 0x10, 0xc20816d200, 0x0, 0x0) /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/godbus/dbus/transport_unix.go:21 +0xc5 io.ReadAtLeast(0x7f7eeae0df58, 0xc20816d200, 0xc2080e28c0, 0x10, 0x10, 0x10, 0x0, 0x0, 0x0) /usr/src/go/src/io/io.go:298 +0xf1 io.ReadFull(0x7f7eeae0df58, 0xc20816d200, 0xc2080e28c0, 0x10, 0x10, 0x0, 0x0, 0x0) /usr/src/go/src/io/io.go:316 +0x6d github.com/godbus/dbus.(*unixTransport).ReadMessage(0xc2081250d0, 0xc208112660, 0x0, 0x0) /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/godbus/dbus/transport_unix.go:85 +0x1bf github.com/godbus/dbus.(*Conn).inWorker(0xc2080daa20) /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/godbus/dbus/conn.go:241 +0x58 created by github.com/godbus/dbus.(*Conn).Auth /go/src/k8s.io/kubernetes/Godeps/_workspace/src/github.com/godbus/dbus/auth.go:118 +0xe84 goroutine 16 [runnable]: k8s.io/kubernetes/pkg/util/iptables.(*runner).dbusSignalHandler(0xc2080d6850, 0x7f7eeae0e028, 0xc20803a100) /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/iptables/iptables.go:525 created by k8s.io/kubernetes/pkg/util/iptables.(*runner).connectToFirewallD /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/iptables/iptables.go:186 +0x7a7 anyone who can help me?
This looks like a bug when using the --cleanup-iptables=true flag in the 1.1.2 release, as I can reproduce a panic when running on a GCE node. I've created kubernetes#18197 on your behalf and this bug will be fixed in the upcoming 1.1.3 release.
PNG File is corruped or not enough memory
Writing my first application using wxWidgets, everything compiled & run fine until I tried to load a few images for my application's toolbar! It always throws: PNG file corrupted by ASCII conversion I've written a simple perl script to convert PNG into compileable source & a header, then modified it a lot while searching for solutions to this problem over the online globe whose source is as follows: #!/usr/bin/perl # Script to convert all PNGs in ./png folder # to C/CPP headers for embedding in apkstudio # binary. # # Written by: Vaibhav Pandey <contact#vaibhavpandey.com> # Declaring used namespaces use Cwd; use File::Basename; # Converts the binary file to C/CPP header sub Convert { my $png = $_[0]; Print("Converting: ${png}"); # Extract base filename from absolute path $name = File::Basename::basename($png, '.png'); # Clear variables & read png file local $/ = undef; open FILE, "${png}" or die "Cannot read: ${png}\n"; $input = <FILE>; close FILE; # Explode input to unsigned char* #characters = unpack "C*", $input; # Write input to output $namespace = uc $name; print HEADER " static const unsigned char* ${name}_png;\n"; print CPP "static const unsigned char _${name}_png[] = {\n "; foreach $character (#characters) { printf CPP '0x%02x', $character; last if $i == $#characters; print CPP ((++$i % 13) ? ', ' : ",\n "); } print CPP "\n};\n"; print CPP "const unsigned char* Embedded::${name}_png = _${name}_png;\n\n"; } # Closes the namespace & class enclosure of the source file sub FooterCpp { print CPP "} // namespace Embedded\n"; print CPP "} // namespace APKStudio\n"; print CPP "} // namespace VPZ\n\n"; } # Closes the namespace & class enclosure of the header file sub FooterHeader { print HEADER "\n};\n\n"; print HEADER "} // namespace Embedded\n"; print HEADER "} // namespace APKStudio\n"; print HEADER "} // namespace VPZ\n\n"; print HEADER "#endif // VPZ_APKSTUDIO_RESOURCE_EMBEDDED\n"; } # Write the namespace & class enclosure to the source file sub HeaderCpp { print CPP "// Generated by resources.pl. Do not modify.\n"; print CPP "#include \"embedded.h\"\n\n"; print CPP "namespace VPZ {\n"; print CPP "namespace APKStudio {\n"; print CPP "namespace Resource {\n\n"; } # Write the namespace & class enclosure to the header file sub HeaderHeader { print HEADER "// Generated by resources.pl. Do not modify.\n"; print HEADER "#ifndef VPZ_APKSTUDIO_RESOURCE_EMBEDDED\n"; print HEADER "#define VPZ_APKSTUDIO_RESOURCE_EMBEDDED\n\n"; print HEADER "#include <wx/bitmap.h>\n\n"; print HEADER "namespace VPZ {\n"; print HEADER "namespace APKStudio {\n"; print HEADER "namespace Resource {\n\n"; print HEADER "class Embedded {\n\n"; print HEADER "public:\n"; } # Prints output to console with an EOL auto-appended sub Print { my $text = $_[0]; print "${text}\n"; } # Scans for PNG files recursively in a folder sub Scan { my $folder = $_[0]; # Check if is not a folder if (! -d $folder) { Print("Not a folder: ${folder}"); return; } Print("Scanning: ${folder}"); # Try to open folder for scanning opendir DIR, $folder or die $!; # Loop through all the files inside while ($entry = readdir DIR) { # Skip . & .. files next if substr($entry, 0, 1) eq "."; my $path = "${folder}/${entry}"; Print("Found: ${path}"); if (-d $path) { # Scan recursively if found a folder ::Scan($path); } elsif (-f $path) { # Continue if not a PNG file next unless $entry =~ m/\.png$/; # Finally convert if is valid ::Convert($path); } } # Close the opened directory handle closedir(DIR); } # Get current working directory $wd = Cwd::getcwd(); # Compute absolute path for C/CPP header my $header = "${wd}/embedded.h"; my $cpp = "${wd}/embedded.cpp"; # Open output stream for writing open HEADER, ">${header}" or die "Cannot write: ${header}\n"; open CPP, ">${cpp}" or die "Cannot write: ${cpp}\n"; HeaderHeader(HEADER); HeaderCpp(CPP); # Fires up the whole thing ::Scan("${wd}/png"); # Close the already open header file FooterHeader(HEADER); FooterCpp(CPP); close CPP; close HEADER; To me, it seems it has read the PNG as text, not in binary mode (I ain't sure yet as I never dealt with either Perl or wxWidgets, and it is my second C++ application ever)! I also tried the offical (it seems as is featured on the website) perl script here, but it is same. Also, I verified the array elements of output by both scripts, mine & the seems-to-be-official one and is same as follows: static const unsigned char _android_png[] = { 0x89, 0x50, 0x4e, 0x47, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x10, 0x08, 0x06, 0x00, 0x00, 0x00, 0x1f, 0xf3, 0xff, 0x61, 0x00, 0x00, 0x00, 0x19, 0x74, 0x45, 0x58, 0x74, 0x53, 0x6f, 0x66, 0x74, 0x77, 0x61, 0x72, 0x65, 0x00, 0x41, 0x64, 0x6f, 0x62, 0x65, 0x20, 0x49, 0x6d, 0x61, 0x67, 0x65, 0x52, 0x65, 0x61, 0x64, 0x79, 0x71, 0xc9, 0x65, 0x3c, 0x00, 0x00, 0x03, 0x69, 0x69, 0x54, 0x58, 0x74, 0x58, 0x4d, 0x4c, 0x3a, 0x63, 0x6f, 0x6d, 0x2e, 0x61, 0x64, 0x6f, 0x62, 0x65, 0x2e, 0x78, 0x6d, 0x70, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x3f, 0x78, 0x70, 0x61, 0x63, 0x6b, 0x65, 0x74, 0x20, 0x62, 0x65, 0x67, 0x69, 0x6e, 0x3d, 0x22, 0xef, 0xbb, 0xbf, 0x22, 0x20, 0x69, 0x64, 0x3d, 0x22, 0x57, 0x35, 0x4d, 0x30, 0x4d, 0x70, 0x43, 0x65, 0x68, 0x69, 0x48, 0x7a, 0x72, 0x65, 0x53, 0x7a, 0x4e, 0x54, 0x63, 0x7a, 0x6b, 0x63, 0x39, 0x64, 0x22, 0x3f, 0x3e, 0x20, 0x3c, 0x78, 0x3a, 0x78, 0x6d, 0x70, 0x6d, 0x65, 0x74, 0x61, 0x20, 0x78, 0x6d, 0x6c, 0x6e, 0x73, 0x3a, 0x78, 0x3d, 0x22, 0x61, 0x64, 0x6f, 0x62, 0x65, 0x3a, 0x6e, 0x73, 0x3a, 0x6d, 0x65, 0x74, 0x61, 0x2f, 0x22, 0x20, 0x78, 0x3a, 0x78, 0x6d, 0x70, 0x74, 0x6b, 0x3d, 0x22, 0x41, 0x64, 0x6f, 0x62, 0x65, 0x20, 0x58, 0x4d, 0x50, 0x20, 0x43, 0x6f, 0x72, 0x65, 0x20, 0x35, 0x2e, 0x30, 0x2d, 0x63, 0x30, 0x36, 0x30, 0x20, 0x36, 0x31, 0x2e, 0x31, 0x33, 0x34, 0x37, 0x37, 0x37, 0x2c, 0x20, 0x32, 0x30, 0x31, 0x30, 0x2f, 0x30, 0x32, 0x2f, 0x31, 0x32, 0x2d, 0x31, 0x37, 0x3a, 0x33, 0x32, 0x3a, 0x30, 0x30, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x22, 0x3e, 0x20, 0x3c, 0x72, 0x64, 0x66, 0x3a, 0x52, 0x44, 0x46, 0x20, 0x78, 0x6d, 0x6c, 0x6e, 0x73, 0x3a, 0x72, 0x64, 0x66, 0x3d, 0x22, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x77, 0x77, 0x77, 0x2e, 0x77, 0x33, 0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x31, 0x39, 0x39, 0x39, 0x2f, 0x30, 0x32, 0x2f, 0x32, 0x32, 0x2d, 0x72, 0x64, 0x66, 0x2d, 0x73, 0x79, 0x6e, 0x74, 0x61, 0x78, 0x2d, 0x6e, 0x73, 0x23, 0x22, 0x3e, 0x20, 0x3c, 0x72, 0x64, 0x66, 0x3a, 0x44, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x20, 0x72, 0x64, 0x66, 0x3a, 0x61, 0x62, 0x6f, 0x75, 0x74, 0x3d, 0x22, 0x22, 0x20, 0x78, 0x6d, 0x6c, 0x6e, 0x73, 0x3a, 0x78, 0x6d, 0x70, 0x52, 0x69, 0x67, 0x68, 0x74, 0x73, 0x3d, 0x22, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x6e, 0x73, 0x2e, 0x61, 0x64, 0x6f, 0x62, 0x65, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x78, 0x61, 0x70, 0x2f, 0x31, 0x2e, 0x30, 0x2f, 0x72, 0x69, 0x67, 0x68, 0x74, 0x73, 0x2f, 0x22, 0x20, 0x78, 0x6d, 0x6c, 0x6e, 0x73, 0x3a, 0x78, 0x6d, 0x70, 0x4d, 0x4d, 0x3d, 0x22, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x6e, 0x73, 0x2e, 0x61, 0x64, 0x6f, 0x62, 0x65, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x78, 0x61, 0x70, 0x2f, 0x31, 0x2e, 0x30, 0x2f, 0x6d, 0x6d, 0x2f, 0x22, 0x20, 0x78, 0x6d, 0x6c, 0x6e, 0x73, 0x3a, 0x73, 0x74, 0x52, 0x65, 0x66, 0x3d, 0x22, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x6e, 0x73, 0x2e, 0x61, 0x64, 0x6f, 0x62, 0x65, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x78, 0x61, 0x70, 0x2f, 0x31, 0x2e, 0x30, 0x2f, 0x73, 0x54, 0x79, 0x70, 0x65, 0x2f, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x52, 0x65, 0x66, 0x23, 0x22, 0x20, 0x78, 0x6d, 0x6c, 0x6e, 0x73, 0x3a, 0x78, 0x6d, 0x70, 0x3d, 0x22, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x6e, 0x73, 0x2e, 0x61, 0x64, 0x6f, 0x62, 0x65, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x78, 0x61, 0x70, 0x2f, 0x31, 0x2e, 0x30, 0x2f, 0x22, 0x20, 0x78, 0x6d, 0x70, 0x52, 0x69, 0x67, 0x68, 0x74, 0x73, 0x3a, 0x4d, 0x61, 0x72, 0x6b, 0x65, 0x64, 0x3d, 0x22, 0x46, 0x61, 0x6c, 0x73, 0x65, 0x22, 0x20, 0x78, 0x6d, 0x70, 0x4d, 0x4d, 0x3a, 0x44, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x49, 0x44, 0x3d, 0x22, 0x78, 0x6d, 0x70, 0x2e, 0x64, 0x69, 0x64, 0x3a, 0x39, 0x41, 0x31, 0x42, 0x43, 0x43, 0x37, 0x43, 0x38, 0x42, 0x31, 0x46, 0x31, 0x31, 0x45, 0x30, 0x41, 0x42, 0x35, 0x33, 0x44, 0x38, 0x45, 0x42, 0x33, 0x45, 0x33, 0x43, 0x42, 0x38, 0x34, 0x31, 0x22, 0x20, 0x78, 0x6d, 0x70, 0x4d, 0x4d, 0x3a, 0x49, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x63, 0x65, 0x49, 0x44, 0x3d, 0x22, 0x78, 0x6d, 0x70, 0x2e, 0x69, 0x69, 0x64, 0x3a, 0x39, 0x41, 0x31, 0x42, 0x43, 0x43, 0x37, 0x42, 0x38, 0x42, 0x31, 0x46, 0x31, 0x31, 0x45, 0x30, 0x41, 0x42, 0x35, 0x33, 0x44, 0x38, 0x45, 0x42, 0x33, 0x45, 0x33, 0x43, 0x42, 0x38, 0x34, 0x31, 0x22, 0x20, 0x78, 0x6d, 0x70, 0x3a, 0x43, 0x72, 0x65, 0x61, 0x74, 0x6f, 0x72, 0x54, 0x6f, 0x6f, 0x6c, 0x3d, 0x22, 0x41, 0x64, 0x6f, 0x62, 0x65, 0x20, 0x50, 0x68, 0x6f, 0x74, 0x6f, 0x73, 0x68, 0x6f, 0x70, 0x20, 0x43, 0x53, 0x33, 0x20, 0x57, 0x69, 0x6e, 0x64, 0x6f, 0x77, 0x73, 0x22, 0x3e, 0x20, 0x3c, 0x78, 0x6d, 0x70, 0x4d, 0x4d, 0x3a, 0x44, 0x65, 0x72, 0x69, 0x76, 0x65, 0x64, 0x46, 0x72, 0x6f, 0x6d, 0x20, 0x73, 0x74, 0x52, 0x65, 0x66, 0x3a, 0x69, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x63, 0x65, 0x49, 0x44, 0x3d, 0x22, 0x75, 0x75, 0x69, 0x64, 0x3a, 0x41, 0x43, 0x31, 0x46, 0x32, 0x45, 0x38, 0x33, 0x33, 0x32, 0x34, 0x41, 0x44, 0x46, 0x31, 0x31, 0x41, 0x41, 0x42, 0x38, 0x43, 0x35, 0x33, 0x39, 0x30, 0x44, 0x38, 0x35, 0x42, 0x35, 0x42, 0x33, 0x22, 0x20, 0x73, 0x74, 0x52, 0x65, 0x66, 0x3a, 0x64, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x49, 0x44, 0x3d, 0x22, 0x75, 0x75, 0x69, 0x64, 0x3a, 0x43, 0x39, 0x44, 0x33, 0x34, 0x39, 0x36, 0x36, 0x34, 0x41, 0x33, 0x43, 0x44, 0x44, 0x31, 0x31, 0x42, 0x30, 0x38, 0x41, 0x42, 0x42, 0x42, 0x43, 0x46, 0x46, 0x31, 0x37, 0x32, 0x31, 0x35, 0x36, 0x22, 0x2f, 0x3e, 0x20, 0x3c, 0x2f, 0x72, 0x64, 0x66, 0x3a, 0x44, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x3e, 0x20, 0x3c, 0x2f, 0x72, 0x64, 0x66, 0x3a, 0x52, 0x44, 0x46, 0x3e, 0x20, 0x3c, 0x2f, 0x78, 0x3a, 0x78, 0x6d, 0x70, 0x6d, 0x65, 0x74, 0x61, 0x3e, 0x20, 0x3c, 0x3f, 0x78, 0x70, 0x61, 0x63, 0x6b, 0x65, 0x74, 0x20, 0x65, 0x6e, 0x64, 0x3d, 0x22, 0x72, 0x22, 0x3f, 0x3e, 0x27, 0x66, 0xfb, 0x9f, 0x00, 0x00, 0x02, 0xcb, 0x49, 0x44, 0x41, 0x54, 0x78, 0xda, 0x8c, 0x52, 0x4b, 0x4f, 0x13, 0x51, 0x14, 0xfe, 0x66, 0x7a, 0xfb, 0x18, 0x68, 0xa7, 0xbc, 0x8a, 0x50, 0x25, 0x22, 0xc6, 0x18, 0x94, 0x44, 0x08, 0x24, 0x36, 0x69, 0xdc, 0xb8, 0x81, 0x44, 0xc5, 0xe8, 0xa2, 0x0b, 0x13, 0x49, 0x64, 0xc1, 0xca, 0x3f, 0x20, 0xee, 0x8d, 0x2b, 0x37, 0xee, 0xba, 0xc0, 0x84, 0xc4, 0x18, 0xd8, 0x68, 0x84, 0x10, 0x1b, 0x34, 0x71, 0x25, 0x4d, 0x28, 0x62, 0x0d, 0xc5, 0xf2, 0x12, 0x79, 0x95, 0x47, 0x4b, 0x81, 0x29, 0xd3, 0x96, 0xf6, 0xb6, 0xe3, 0x99, 0xb6, 0x20, 0x89, 0x0b, 0xbd, 0x93, 0x93, 0x7b, 0xce, 0x9c, 0xef, 0x7c, 0xf7, 0x9c, 0x2f, 0x47, 0xc0, 0x90, 0x03, 0xa5, 0x13, 0x40, 0x5e, 0xeb, 0x06, 0xcf, 0x47, 0x60, 0x10, 0xf0, 0xd7, 0xc9, 0x69, 0x00, 0x13, 0x9d, 0x10, 0x85, 0xf7, 0x14, 0x75, 0xe8, 0xbf, 0x34, 0xcf, 0x0e, 0x18, 0x34, 0xad, 0x08, 0xd0, 0xd0, 0x4d, 0x04, 0x7a, 0xb2, 0x9f, 0xfc, 0x3b, 0x74, 0x5f, 0x21, 0xb3, 0x91, 0x25, 0xc8, 0x66, 0xc9, 0x46, 0x28, 0xff, 0x8c, 0xee, 0x6e, 0x9c, 0xe2, 0x67, 0xd8, 0x4d, 0x15, 0x3d, 0x51, 0x88, 0xc0, 0x64, 0x78, 0x23, 0x5b, 0xed, 0xcf, 0x3b, 0x9a, 0x3b, 0xce, 0x9f, 0xad, 0x39, 0x27, 0x1b, 0x99, 0x91, 0x65, 0xb2, 0x19, 0xbe, 0x1e, 0x5b, 0x6f, 0xfd, 0x1a, 0x0e, 0xb8, 0x95, 0x43, 0xe5, 0x35, 0x32, 0xb9, 0x08, 0x11, 0x9d, 0x10, 0x88, 0x27, 0x9e, 0x86, 0x9b, 0x95, 0x16, 0xb9, 0xe7, 0x46, 0xb3, 0xbb, 0x65, 0x77, 0x71, 0xa3, 0x6a, 0x63, 0xf9, 0x17, 0x8b, 0xc7, 0xe3, 0x58, 0x59, 0x5a, 0x62, 0xd1, 0xc5, 0xb5, 0x2a, 0x22, 0x6d, 0xb1, 0x59, 0xca, 0x7b, 0x74, 0xdc, 0xe9, 0xc9, 0x44, 0x58, 0x18, 0x90, 0xcd, 0x01, 0x19, 0xfe, 0xc8, 0x75, 0xcd, 0xdd, 0xb8, 0x1c, 0x9e, 0x37, 0x7e, 0xbb, 0xe7, 0xc7, 0xf4, 0x97, 0x09, 0x6c, 0x45, 0xb6, 0x10, 0x9c, 0xf0, 0x63, 0xe6, 0xfe, 0x24, 0xe6, 0x7f, 0xcc, 0x18, 0x2f, 0xb4, 0x5c, 0x6c, 0xd4, 0x71, 0x05, 0xbc, 0x5e, 0x57, 0x18, 0xc1, 0x6a, 0x02, 0x8c, 0xd4, 0xc8, 0x66, 0xa2, 0xa9, 0x52, 0xae, 0xb6, 0x2a, 0x67, 0x14, 0x3c, 0xfe, 0xfe, 0x04, 0x35, 0x0d, 0x0d, 0xd8, 0xde, 0xde, 0x86, 0xb9, 0xc6, 0x0e, 0xcf, 0x64, 0x2f, 0x72, 0xb5, 0x06, 0xa4, 0x6d, 0x19, 0x2b, 0x8e, 0x78, 0x13, 0xea, 0x49, 0x1a, 0xf3, 0x31, 0x81, 0x86, 0x3e, 0x0a, 0xfa, 0x90, 0xc9, 0x23, 0x14, 0x0a, 0x89, 0xb1, 0x58, 0x0c, 0xb3, 0xe1, 0x30, 0x32, 0xaa, 0x0a, 0x35, 0x1a, 0x85, 0x89, 0x31, 0x7c, 0x3a, 0x18, 0x83, 0x5c, 0x5d, 0x0d, 0x73, 0x82, 0x89, 0x84, 0x33, 0x13, 0x3e, 0x40, 0x75, 0x5e, 0xaa, 0xf7, 0x0a, 0x18, 0xac, 0x0c, 0xf4, 0x76, 0xf5, 0xb6, 0x07, 0x83, 0x41, 0xb4, 0xb5, 0xb6, 0x21, 0x15, 0x4f, 0xc2, 0x94, 0x65, 0x64, 0x46, 0xa8, 0x8a, 0x8a, 0xb4, 0x90, 0x46, 0x42, 0x3a, 0xc4, 0xbe, 0x4d, 0x81, 0x22, 0x27, 0x50, 0x16, 0x2a, 0x43, 0xf4, 0x6a, 0x14, 0xab, 0x1f, 0x57, 0xa7, 0xb4, 0x87, 0xf1, 0x0e, 0x86, 0x64, 0x16, 0x9c, 0xf3, 0x42, 0x3b, 0xc9, 0x54, 0x12, 0x6b, 0xb1, 0x35, 0x24, 0x93, 0x49, 0xa4, 0xd4, 0x14, 0xa2, 0xd4, 0x01, 0x67, 0x94, 0xab, 0xa4, 0x64, 0x35, 0x20, 0x19, 0x25, 0x94, 0xd1, 0xc7, 0x73, 0xf4, 0x8f, 0xea, 0x8a, 0x22, 0x96, 0x9c, 0xbd, 0xbd, 0x3d, 0xfc, 0xcf, 0x39, 0xc1, 0x95, 0xea, 0x18, 0xd2, 0xdc, 0x3b, 0xf8, 0xea, 0x65, 0x9f, 0x1e, 0xb8, 0x5c, 0xae, 0xf6, 0x7f, 0x11, 0xfc, 0x5c, 0x98, 0x9e, 0xc2, 0x02, 0x39, 0x82, 0xe0, 0x2d, 0x89, 0xa8, 0x79, 0x49, 0x29, 0x2f, 0xea, 0x6c, 0xfe, 0xe8, 0xee, 0xb6, 0x22, 0x49, 0x92, 0xac, 0xe9, 0xdb, 0x99, 0xa7, 0x96, 0x25, 0xa9, 0x38, 0x82, 0x85, 0x90, 0x12, 0xc0, 0xe3, 0xd9, 0x03, 0x38, 0x2b, 0x38, 0xa2, 0x87, 0x2e, 0x64, 0x79, 0xa9, 0x03, 0xfd, 0xd8, 0xca, 0x81, 0x7d, 0xee, 0x1d, 0x7f, 0x37, 0xe2, 0x81, 0x9a, 0xc1, 0x65, 0xf7, 0xf5, 0xce, 0xb9, 0x31, 0xbf, 0x0f, 0x07, 0x3a, 0x42, 0x44, 0xed, 0xdd, 0x4b, 0x9d, 0x3b, 0xbe, 0x15, 0x1f, 0x64, 0x03, 0xe9, 0xc1, 0x86, 0x51, 0x5e, 0x46, 0x78, 0xa5, 0x44, 0xf0, 0x34, 0xaf, 0x4f, 0x06, 0x74, 0x99, 0x07, 0x20, 0xf3, 0x01, 0xd8, 0xb5, 0x51, 0x87, 0xc3, 0x81, 0xb9, 0x34, 0x38, 0xed, 0xfc, 0x6d, 0x5a, 0xdb, 0x51, 0x66, 0xa7, 0x77, 0x14, 0x9e, 0xc3, 0x46, 0xfe, 0x16, 0x0e, 0xe8, 0x65, 0xdf, 0x51, 0x71, 0x9e, 0xfe, 0xe3, 0x0e, 0x40, 0xd2, 0x7e, 0x38, 0x92, 0x0b, 0xa2, 0x3e, 0x40, 0x05, 0x63, 0xfa, 0x76, 0xa2, 0x02, 0x43, 0x70, 0xc2, 0xa3, 0x15, 0xe3, 0x54, 0xde, 0x8e, 0x61, 0xee, 0x2c, 0x2c, 0x3d, 0xd1, 0x91, 0xa9, 0x7f, 0x46, 0xd0, 0x09, 0x80, 0x3a, 0x32, 0x03, 0xd6, 0x30, 0xfe, 0xf9, 0xc5, 0x5b, 0x1b, 0x22, 0x18, 0xa7, 0xb8, 0x5e, 0xbf, 0x57, 0xbd, 0x41, 0x1b, 0x36, 0x85, 0x62, 0x5c, 0x50, 0x07, 0xb9, 0x63, 0x82, 0xdf, 0x02, 0x0c, 0x00, 0xbc, 0x10, 0x2e, 0x3f, 0x7d, 0xcf, 0x4d, 0x1e, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82 }; The code where it throws error is just the simple block of a couple lines as mentioned on the website: wxMemoryInputStream inStream(Resource::Embedded::android_png, sizeof(Resource::Embedded::android_png)); wxBitmap image(wxImage(inStream, wxBITMAP_TYPE_PNG)); Also, if it can help...I am running ActiveState's distribution of Perl specifically the 5.18.2.1802 version on a 64-bit AMD machine with Windows 8.1. I have strong doubt on ActiveState Perl's file handling implementations though strictly need expert advisory if anyone has got time to help me. I'll be really grateful.
Use the three-argument open in your Perl script. Then you can add the :raw layer: open FILE, '<:raw', $png or die "Cannot read: ${png}: $!\n";
Problem solved! The issue was with the CPP code, particularly in member variable initialisations. Sorry for bothering!
any good alternative to Iconv library for encoding conversion?
i was using Iconv library on Ruby to convert encoding from UTF-8 to UTF-32, UTF-16 etc and it was quite good. However, I do see an issue when converting from Big5 to UTF-8 -- an exception is thrown for invalid sequence... and the problem goes away when it is converting from CP950 to UTF-8, of which CP950 is essentially Big5... so I wonder if there is another good alternative besides using Iconv? Or is the CP950 a better version of Big5?
Have a look at ICU, it's a library that does character conversions among other things. On the other hand, the other answers suggest you might need to examine your encodings more carefully.
Although Big5 and CP950 are almost the same there are differences. On the Unicode website there are reference files for converting different encodings to Unicode, you will see that Big5 and CP950 are different. My experience of Iconv has been good - I suspect its behavior may be correct. Some of the characters found in CP950 but not Big5 have the hex values: 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2A, 0x2B, 0x2C, 0x2D, 0x2E, 0x2F, 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, 0x3D, 0x3E, 0x3F, 0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x4A, 0x4B, 0x4C, 0x4D, 0x4E, 0x4F, 0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5A, 0x5B, 0x5C, 0x5D, 0x5E, 0x5F, 0x60, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x6A, 0x6B, 0x6C, 0x6D, 0x6E, 0x6F, 0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7A, 0x7B, 0x7C, 0x7D, 0x7E, 0x7F, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8A, 0x8B, 0x8C, 0x8D, 0x8E, 0x8F, 0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9A, 0x9B, 0x9C, 0x9D, 0x9E, 0x9F, 0xA0, 0xA1, 0xA15A, 0xA1C3, 0xA1C5, 0xA1FE, 0xA240, 0xA3E1, 0xA2CC, 0xA2CE. If any of your input contains these values, then the file is not valid Big5.
There are many many big5 variants. CP950 is just one of them. http://www.moztw.org/docs/big5/ For big5, I would suggest use "big5-2003", which is official updated version.
Alesis QS MIDI Sysex Data Conversion
My aim is to convert a stream of byte code sent from an Alesis synthesizer to a human readable format. I need to be able to take a "Program Dump" and read the 10 character string that makes up the patch name. In order to receive the "Program Dump" from the synth, I sent the synth the following command via MIDI-OX: F0 00 00 0E 0E 01 73 F7 I requested that it send me the dump for program 73. I received this: F0 00 00 0E 0E 00 73 00 60 24 0B 27 27 01 64 1E 19 19 05 23 19 1E 2A 41 0D 23 46 19 1E 06 00 47 0D 23 30 6C 18 63 30 6C 18 40 3F 0A 67 1B 16 20 40 00 60 18 00 18 06 05 0C 2B 41 13 70 05 30 40 31 63 70 05 00 40 31 63 70 05 00 40 31 63 00 4C 2A 51 00 46 7F 78 18 40 0F 40 31 40 31 04 30 0C 00 30 6C 03 30 3C 0F 00 00 05 0A 0F 14 19 1E 23 28 2D 72 00 76 34 3C 54 42 19 46 0C 33 3C 0C 00 0E 1B 46 60 58 31 46 61 58 31 00 7F 14 4E 37 6C 74 13 00 40 31 00 30 0C 0A 18 56 02 27 60 0B 60 00 63 46 61 0B 00 00 63 46 61 0B 00 00 63 46 01 18 55 22 01 0C 7F 71 31 00 1F 00 63 00 63 08 60 18 00 60 58 07 60 18 1E 00 00 0A 14 1E 28 32 3C 46 50 5A 64 01 0C 2D 15 29 05 36 0C 19 66 78 18 00 1C 36 0C 41 31 63 0C 43 31 63 00 7E 29 1C 6F 58 00 01 02 00 63 00 60 18 14 30 2C 05 4E 40 17 40 01 46 0D 43 17 00 00 46 0D 43 17 00 00 46 0D 03 30 2A 45 02 18 7E 63 63 00 3E 00 46 01 46 11 40 31 00 40 31 0F 40 71 3D 00 00 14 28 3C 50 64 78 0C 21 35 49 03 58 4C 71 31 1C 6C 18 32 4C 71 31 00 38 6C 18 02 63 46 19 06 63 46 01 7C 53 00 60 18 53 37 6C 70 0D 03 40 31 28 60 58 0A 1C 01 2F 00 03 0C 1B 06 2F 00 00 0C 1B 06 2F 00 00 0C 1B 06 60 54 0A 05 30 7C 47 47 01 7C 00 0C 03 0C 23 00 63 00 00 63 1E 3C 63 18 00 00 28 50 78 20 49 71 19 42 6A 12 07 F7 MIDI-OX told me that it received 408 bytes. This jives with the specification: "There are 400 data bytes sent for a single program dump, which corresponds to 350 bytes of program data. With the header, the total number of bytes transmitted with a program dump is 408. The location of each parameter within a program dump is shown in the next section." The "Program Dump" should be composed of these values: F0 00 00 0E 0E 00 <program#> <data> F7 That means the data should begin with "00 60" and end with "07 F7". Now I should be able to convert these 400 bytes to the "350 bytes of packed parameter data" for this program. Following the "Program Data Format", 10 digits of the program name should be located within the packed data somewhere. Patch 73 is called either "BlowDeTune" or "PanBristle", not totally sure if it starts at 0 or 1. So how do you go about make the conversion? Page 1 of the specification gives the transmission format, but I don't understand how to unpack it. Can anyone help? The Alesis QS MIDI Sysex Specification is here: http://www.midiworld.com/quadrasynth/qs_swlib/qs678r.pdf MIDI-OX can be found here: http://www.midiox.com/
You are lucky, because some years ago I played a bit with Midi (with my Atari ST 520) so I had enough interest in the topic to investigate a bit... For the record, I found the MIDI System Exclusive Message format, in accordance with the reference you give for your synth. I first thought the packing algorithm was described in this page, but after implementing its decoding and founding garbage, I saw I was wrong... I will give this code just in case it could be useful to you elsewhere... This first try was useful because when I re-read the spec in the PDF file, I understood the A7 to G0 symbols were actually bits... Data is "packed" because Midi non-control words must be 7bit clean (high bit always unset). They take 7 bytes of raw data, see it as a big word of 56 bits, and split that word every 7 bits, leaving the high bit always at 0: Original data (using a different notation): 0 - b07 b06 b05 b04 b03 b02 b01 b00 1 - b17 b16 b15 b14 b13 b12 b11 b10 2 - b27 b26 b25 b24 b23 b22 b21 b20 3 - b37 b36 b35 b34 b33 b32 b31 b30 4 - b47 b46 b45 b44 b43 b42 b41 b40 5 - b57 b56 b55 b54 b53 b52 b51 b50 6 - b67 b66 b65 b64 b63 b62 b61 b60 Transmitted/encoded data: 0 - 0 b06 b05 b04 b03 b02 b01 b00 1 - 0 b15 b14 b13 b12 b11 b10 b07 2 - 0 b24 b23 b22 b21 b20 b17 b16 3 - 0 b33 b32 b31 b30 b27 b26 b25 4 - 0 b42 b41 b40 b37 b36 b35 b34 5 - 0 b51 b50 b47 b46 b45 b44 b43 6 - 0 b60 b57 b56 b55 b54 b53 b52 7 - 0 b67 b66 b65 b64 b63 b62 b61 So we have: 0 - 00000000 0x00 1 - 01100000 0x60 2 - 00100100 0x24 3 - 00001011 0x0B 4 - 00100111 0x27 5 - 00100111 0x27 6 - 00000001 0x01 7 - 01100100 0x64 0 - 00011110 0x1E 1 - 00011001 0x19 2 - 00011001 0x19 3 - 00000101 0x05 4 - 00100011 0x23 5 - 00011001 0x19 6 - 00011110 0x1E 7 - 00101010 0x2A and once decoded, we should have: 0 - 00000000 0x00 1 - 00110000 0x30 2 - 01101001 0x69 3 - 01110001 0x71 4 - 00111010 0x3A 5 - 00000101 0x05 6 - 11001000 0xC8 0 - 10011110 0x9E 1 - 01001100 0x4C 2 - 10100110 0xA6 3 - 00110000 0x30 4 - 11001010 0xCA 5 - 01111000 0x78 6 - 01010100 0x54 I believe I decoded correctly the data, but still have garbage (ie. non readable strings)... Perhaps you will see a logic error in my code, which might be a starting point anyway. I saw that MIDI-OX can be scripted with WSH, so I wrote a JS script which I ran with WSH, with output on console: var midiData = [ 0xF0, 0x00, 0x00, 0x0E, 0x0E, 0x00, 0x73, 0x00, 0x60, 0x24, 0x0B, 0x27, 0x27, 0x01, 0x64, 0x1E, 0x19, 0x19, 0x05, 0x23, 0x19, 0x1E, 0x2A, 0x41, 0x0D, 0x23, 0x46, 0x19, 0x1E, 0x06, 0x00, 0x47, 0x0D, 0x23, 0x30, 0x6C, 0x18, 0x63, 0x30, 0x6C, 0x18, 0x40, 0x3F, 0x0A, 0x67, 0x1B, 0x16, 0x20, 0x40, 0x00, 0x60, 0x18, 0x00, 0x18, 0x06, 0x05, 0x0C, 0x2B, 0x41, 0x13, 0x70, 0x05, 0x30, 0x40, 0x31, 0x63, 0x70, 0x05, 0x00, 0x40, 0x31, 0x63, 0x70, 0x05, 0x00, 0x40, 0x31, 0x63, 0x00, 0x4C, 0x2A, 0x51, 0x00, 0x46, 0x7F, 0x78, 0x18, 0x40, 0x0F, 0x40, 0x31, 0x40, 0x31, 0x04, 0x30, 0x0C, 0x00, 0x30, 0x6C, 0x03, 0x30, 0x3C, 0x0F, 0x00, 0x00, 0x05, 0x0A, 0x0F, 0x14, 0x19, 0x1E, 0x23, 0x28, 0x2D, 0x72, 0x00, 0x76, 0x34, 0x3C, 0x54, 0x42, 0x19, 0x46, 0x0C, 0x33, 0x3C, 0x0C, 0x00, 0x0E, 0x1B, 0x46, 0x60, 0x58, 0x31, 0x46, 0x61, 0x58, 0x31, 0x00, 0x7F, 0x14, 0x4E, 0x37, 0x6C, 0x74, 0x13, 0x00, 0x40, 0x31, 0x00, 0x30, 0x0C, 0x0A, 0x18, 0x56, 0x02, 0x27, 0x60, 0x0B, 0x60, 0x00, 0x63, 0x46, 0x61, 0x0B, 0x00, 0x00, 0x63, 0x46, 0x61, 0x0B, 0x00, 0x00, 0x63, 0x46, 0x01, 0x18, 0x55, 0x22, 0x01, 0x0C, 0x7F, 0x71, 0x31, 0x00, 0x1F, 0x00, 0x63, 0x00, 0x63, 0x08, 0x60, 0x18, 0x00, 0x60, 0x58, 0x07, 0x60, 0x18, 0x1E, 0x00, 0x00, 0x0A, 0x14, 0x1E, 0x28, 0x32, 0x3C, 0x46, 0x50, 0x5A, 0x64, 0x01, 0x0C, 0x2D, 0x15, 0x29, 0x05, 0x36, 0x0C, 0x19, 0x66, 0x78, 0x18, 0x00, 0x1C, 0x36, 0x0C, 0x41, 0x31, 0x63, 0x0C, 0x43, 0x31, 0x63, 0x00, 0x7E, 0x29, 0x1C, 0x6F, 0x58, 0x00, 0x01, 0x02, 0x00, 0x63, 0x00, 0x60, 0x18, 0x14, 0x30, 0x2C, 0x05, 0x4E, 0x40, 0x17, 0x40, 0x01, 0x46, 0x0D, 0x43, 0x17, 0x00, 0x00, 0x46, 0x0D, 0x43, 0x17, 0x00, 0x00, 0x46, 0x0D, 0x03, 0x30, 0x2A, 0x45, 0x02, 0x18, 0x7E, 0x63, 0x63, 0x00, 0x3E, 0x00, 0x46, 0x01, 0x46, 0x11, 0x40, 0x31, 0x00, 0x40, 0x31, 0x0F, 0x40, 0x71, 0x3D, 0x00, 0x00, 0x14, 0x28, 0x3C, 0x50, 0x64, 0x78, 0x0C, 0x21, 0x35, 0x49, 0x03, 0x58, 0x4C, 0x71, 0x31, 0x1C, 0x6C, 0x18, 0x32, 0x4C, 0x71, 0x31, 0x00, 0x38, 0x6C, 0x18, 0x02, 0x63, 0x46, 0x19, 0x06, 0x63, 0x46, 0x01, 0x7C, 0x53, 0x00, 0x60, 0x18, 0x53, 0x37, 0x6C, 0x70, 0x0D, 0x03, 0x40, 0x31, 0x28, 0x60, 0x58, 0x0A, 0x1C, 0x01, 0x2F, 0x00, 0x03, 0x0C, 0x1B, 0x06, 0x2F, 0x00, 0x00, 0x0C, 0x1B, 0x06, 0x2F, 0x00, 0x00, 0x0C, 0x1B, 0x06, 0x60, 0x54, 0x0A, 0x05, 0x30, 0x7C, 0x47, 0x47, 0x01, 0x7C, 0x00, 0x0C, 0x03, 0x0C, 0x23, 0x00, 0x63, 0x00, 0x00, 0x63, 0x1E, 0x3C, 0x63, 0x18, 0x00, 0x00, 0x28, 0x50, 0x78, 0x20, 0x49, 0x71, 0x19, 0x42, 0x6A, 0x12, 0x07, 0xF7 ]; // Show original data DumpData(midiData, 16); var headerLength = 7; // Bytes to skip var resultData = new Array(); var decodedByteCount = 0; // Number of expanded bytes in result var cumulator = 0; var bitCount = 0; for (var i = headerLength; // Skip header i < midiData.length - 1; // Omit EOF i++) { var rank = (i - headerLength) % 8; // We split the data in runs of 8 bytes // We cumulate the bits of these runs (less the high bit) to make a big word of 56 bits /* cumulator |= midiData[i] << (7 * rank); if (rank == 7) // End of the run { // Split the cumulator in 7 bytes for (var j = 0; j < 7; j++) { var shift = 8 * j; var byte = (cumulator & (0xFF << shift)) >> shift; WScript.StdOut.Write(ByteToHex(byte) + ' '); resultData[decodedByteCount++] = byte; } cumulator = 0; // Reset the buffer } */ // Actually, we cannot do that, because JS' bit arithmetic seems to be limited to signed 32 bits! // So I get the bytes out as soon as they are complete. // Somehow, it is more elegant anyway (but reflects less the original algorithm). cumulator |= midiData[i] << bitCount; bitCount += 7; //~ WScript.StdOut.Write((i - 7) + ':' + ByteToHex(midiData[i]) + ' (' + bitCount + ') ' + DecimalToHex(cumulator) + '\n'); if (bitCount >= 8) { var byte = cumulator & 0xFF; bitCount -= 8; cumulator >>= 8; resultData[decodedByteCount++] = byte; //~ WScript.StdOut.Write((i - 7) + ':' + ByteToHex(midiData[i]) + ' (' + bitCount + ') ' + DecimalToHex(cumulator) + ' > ' + ByteToHex(byte) + '\n'); } } DumpData(resultData, 14); The utility routines: function DumpData(data, lineLength) { WScript.StdOut.Write("Found " + data.length + " bytes\n"); var txt = ''; for (var i = 0; i < data.length; i++) { var rd = data[i]; if (rd > 31) { txt += String.fromCharCode(rd); } else { txt += '.'; } WScript.StdOut.Write(ByteToHex(rd) + ' '); if ((i+1) % lineLength == 0) { WScript.StdOut.Write(' ' + txt + '\n'); txt = ''; } } WScript.StdOut.Write(' ' + txt + '\n'); } function NibbleToHex(halfByte) { return String.fromCharCode(halfByte < 10 ? halfByte + 48 : // 0 to 9 halfByte + 55); // A to F } function ByteToHex(dec) { var h = (dec & 0xF0) >> 4; var l = dec & 0x0F; return NibbleToHex(h) + NibbleToHex(l); } function DecimalToHex(dec) { var result = ''; do { result = ByteToHex(dec & 0xFF) + result; dec >>= 8; } while (dec > 0); return result; } Output: Found 350 bytes 00 30 69 71 3A 05 C8 9E 4C A6 30 CA 78 54 .0iq:.ÈL¦0ÊxT C1 C6 C8 98 F1 18 00 C7 C6 08 C6 C6 8C 61 ÁÆÈñ..ÇÆ.ÆÆa 6C 0C F0 A7 38 6F 2C 20 20 00 8C 01 60 0C l.ð§8o, ..`. 05 C6 2A 38 81 17 60 C0 D8 18 5E 00 00 63 .Æ*8.`ÀØ.^..c 63 78 01 00 8C 8D 01 4C 55 14 60 FC E3 31 cx...LU.`üã1 C0 07 30 06 8C 11 60 0C 00 8C 3D 80 F1 1E À.0..`..=ñ. 00 40 41 F1 A0 64 3C 23 54 4B 0E B0 D3 78 .#Añ d<#TK.°Óx 54 61 C6 C8 98 F1 18 00 C7 C6 08 C6 C6 8C TaÆÈñ..ÇÆ.ÆÆ 61 6C 0C F0 A7 38 6F 6C FA 04 00 8C 01 60 al.ð§8olú...` 0C 05 C6 2A 38 81 17 60 C0 D8 18 5E 00 00 ..Æ*8.`ÀØ.^.. 63 63 78 01 00 8C 8D 01 4C 55 14 60 FC E3 ccx...LU.`üã 31 C0 07 30 06 8C 11 60 0C 00 8C 3D 80 31 1À.0..`..=1 1E 00 40 41 F1 A0 64 3C 23 54 4B 0E 30 5A ..#Añ d<#TK.0Z 95 54 C1 C6 C8 98 F1 18 00 C7 C6 08 C6 C6 TÁÆÈñ..ÇÆ.ÆÆ 8C 61 6C 0C F0 A7 38 6F 2C 20 20 00 8C 01 al.ð§8o, .. 60 0C 05 C6 2A 38 81 17 60 C0 D8 18 5E 00 `..Æ*8.`ÀØ.^. 00 63 63 78 01 00 8C 8D 01 4C 55 14 60 FC .ccx...LU.`ü E3 31 C0 07 30 06 8C 11 60 0C 00 8C 3D 80 ã1À.0..`..= F1 1E 00 40 41 F1 A0 64 3C 23 54 4B 0E B0 ñ..#Añ d<#TK.° CC 78 8C C3 C6 C8 98 F1 18 00 C7 C6 08 C6 ÌxÃÆÈñ..ÇÆ.Æ C6 8C 61 6C 0C F0 A7 00 30 66 7A 63 C3 1B Æal.ð§.0fzcÃ. 03 60 0C 05 C6 2A 38 81 17 60 C0 D8 18 5E .`..Æ*8.`ÀØ.^ 00 00 63 63 78 01 00 8C 8D 01 4C 55 14 60 ..ccx...LU.` FC E3 31 C0 07 30 06 8C 11 60 0C 00 8C 3D üã1À.0..`..= BC 31 06 00 40 41 F1 A0 64 3C 23 54 4B 0E ¼1..#Añ d<#TK. And just in case, the other unpacking algorithm: // Here the 8 bits of 7 bytes of raw data are coded as 7 bytes of data stripped off of the high bit, // while the stripped bits are grouped in the first byte of the data run. // In other words, when we have a run of 8 bytes, the first one groups the high bits of the 7 next bytes. // Information found at http://crystal.apana.org.au/ghansper/midi_introduction/file_dump.html var headerLength = 7; var resultData = new Array(); var decodedByteCount = 0; // Number of expanded bytes in result var runCount = -1; // Number of runs in the encoded data for (var i = headerLength; // Skip header i < midiData.length - 1; // Omit EOF i++) { var rank = (i - headerLength) % 8; // We split the data in runs of 8 bytes if (rank == 0) // Start of the run { // Get the high bits var highBits = midiData[i]; runCount++; //~ WScript.StdOut.Write(runCount + ' > ' + (i - 7) + ' >> ' + ByteToHex(highBits) + '\n'); } else { resultData[decodedByteCount++] = midiData[i] | ((highBits & (1 << (7 - rank))) << rank); //~ WScript.StdOut.Write((i - 7) + ' >> ' + ByteToHex(midiData[i]) + ' > ' + //~ ByteToHex(midiData[i] | ((highBits & (1 << (7 - rank))) << rank)) + '\n'); } }
Thanks to your great work I came up with this as the pack algorithm. It seems that the Alesis uses the same schema as the Moog Voyager. packSysex : function(midiData) { var header = [0xF0, 0x04, 0x01, 0x00, 0x03, 0x00]; //Voyager Single Preset Dump. var resultData = new Array(); var packedByteCount = 0; var bitCount = 0; var thisByte; var packedByte; var nextByte = 0x0; for (var i = 0; i <= midiData.length; i++) { thisByte = midiData[i]; packedByte = ((thisByte << bitCount) | nextByte) & 0x7F; nextByte = midiData[i] >> (7-bitCount); resultData[packedByteCount++] = packedByte; bitCount++; if(bitCount >= 7) { bitCount = 0; //Fill last byte packedByte = nextByte & 0x7F; resultData[packedByteCount++] = packedByte; nextByte = 0x0; } } resultData[packedByteCount++] = 0xF7; resultData = header.concat(resultData); return resultData; },