Hi I have scraped a large number of midi files off the internet.
I am using them for training material to train a generative adversarial network. I find that many midi files conform to the midi standard but then I run into issues with midi meta events with values of FF11 and FF10 . I have looked up the midi specification from several sources and have never found midi meta events defined in this way. Here is the hex of a midi track event with some of the offending values:
4D 54 72 6B 00 00 1A 8D 00 FF 03 0D 47 75 69 74
61 72 20 44 41 44 47 41 44 00 FF 10 08 00 00 3E
39 37 32 2D 26 00 C0 19 00 C1 19 00 B0 65 00 00
B0 64 00 00 B0 06 02 00 B0 65 7F 00 B0 64 7F 00
E0 00 40 00 B1 65 00 00 B1 64 00 00 B1 06 02 00
B1 65 7F 00 B1 64 7F 00 E1 00 40 00 B0 0A 3F 00
B1 0A 3F 00 B0 5D 10 00 B0 5B 1E 00 B1 5D 10 00
B1 5B 1E 81 69 FF 11 01 00 00 90 3E 51 08 FF 11
I cant seem to find any information whatsoever on these values even though these midi files play over timidity and other midi player software perfectly. Can anyone point me to some information about them and what they mean? any help would be greatly , greatly appreciated. :-) resolving this issue would be a service to the miriad of people who are trying to use the python-midi library to train tensorflow models and this I am sure is only a fraction of the people who would be effected.
The SMF specification says:
As with chunks, future meta-events may be designed which may not be known to existing programs, so programs must properly ignore meta-events which they do not recognize, and indeed, should expect to see them.
I am not aware of any published extension that defines values 10h or 11h; it's likely that some sequencer uses these for its own purposes, and violated the specification by not using type 7F for that.
Related
As part of installation of linux, I would like to set the "console device properties"(example, console=ttyS0,115200n1) via the kernel cmdline for Intel based platform.
There is No VGA console, only serial consoles via COM interface.
On these systems BIOS already has the required settings to interact using the appropriate serial port.
I see that EFI has variables ConIn, ConOut, ConErr which I am able to see from /sys/firmware/efi but unable to decode the contents of it.
Is it possible to identify which COM port is being used by the BIOS by examining the efi variables.
Example, of the EFI var on my box.
root#linux:~# efivar -p -n 8be4df61-93ca-11d2-aa0d-00e098032b8c-ConOut
GUID: 8be4df61-93ca-11d2-aa0d-00e098032b8c
Name: "ConOut"
Attributes:
Non-Volatile
Boot Service Access
Runtime Service Access
Value:
00000000 02 01 0c 00 d0 41 03 0a 00 00 00 00 01 01 06 00 |.....A..........|
00000010 00 1a 03 0e 13 00 00 00 00 00 00 c2 01 00 00 00 |................|
00000020 00 00 08 01 01 03 0a 18 00 9d 9a 49 37 2f 54 89 |...........I7/T.|
00000030 4c a0 26 35 da 14 20 94 e4 01 00 00 00 03 0a 14 |L.&5.. .........|
00000040 00 53 47 c1 e0 be f9 d2 11 9a 0c 00 90 27 3f c1 |.SG..........'?.|
00000050 4d 7f 01 04 00 02 01 0c 00 d0 41 03 0a 00 00 00 |M.........A.....|
00000060 00 01 01 06 00 00 1f 02 01 0c 00 d0 41 01 05 00 |............A...|
00000070 00 00 00 03 0e 13 00 00 00 00 00 00 c2 01 00 00 |................|
00000080 00 00 00 08 01 01 03 0a 18 00 9d 9a 49 37 2f 54 |............I7/T|
00000090 89 4c a0 26 35 da 14 20 94 e4 01 00 00 00 03 0a |.L.&5.. ........|
000000a0 14 00 53 47 c1 e0 be f9 d2 11 9a 0c 00 90 27 3f |..SG..........'?|
000000b0 c1 4d 7f ff 04 00 |.M.... |
root#linux:~#
The contents of the ConOut variable are described in the UEFI specification - current version (2.8B):
3.3 - globally defined variables:
| Name | Attribute | Description |
|---------|------------|------------------------------------------------|
| ConOut | NV, BS, RT | The device path of the default output console. |
For information about device paths, we have:
10 - Protocols — Device Path Protocol:
Apart from the initial description of device paths, table 44 shows you the Generic Device Path Node structure, from which we can start decoding the contents of the variable.
The type of the first node is 0x02, telling us this node describes an ACPI device path, of 0x000c bytes length. Now jump down to 10.3.3 - ACPI Device Path and table 52, which tells us 1) that this is the right table (subtype 0x01) and 2) that the default ConOut has a _HID of 0x0a03410d and a _UID of 0.
The next node has a type of 0x01 - a Hardware Device Path, described further in 10.3.2, in this case table 46 (SubType is 0x01) for a PCI device path.
The next node describes a Messaging Device Path of type UART and so on...
Still, this only tells you what UEFI considers to be its default console, SPCR is what an operating system is supposed to be looking at for serial consoles. Unfortunately, on X86 the linux kernel handily ignores SPCR apart from for earlycon. I guess this is what you're trying to work around. It might be good to start some discussion on kernel development lists about whether to fix that and have X86 work like ARM64.
In my case since I know that console port is a "Serial IOPORT",
I could get the details now as follows.
a. Get hold of the /sys/firmware/acpi/tables/SPC table.
b. Read the Address offset 44-52. Actually one the last two bytes suffice.
Reference:
a. https://learn.microsoft.com/en-us/windows-hardware/drivers/serports/serial-port-console-redirection-table states that
Base Address 12 40
The base address of the Serial Port register set described using the ACPI Generic Address Structure.
0 = console redirection disabled
Note:
COM1 (0x3F8) would be:
Integer Form: 0x 01 08 00 00 00000000000003F8
Viewed in Memory: 0x01080000F803000000000000
COM2 (Ox2F8) would be:
Integer Form: 0x 01 08 00 00 00000000000002F8
Viewed in Memory: 0x01080000F802000000000000
I like to modify config files directly (like .gitignore and .git/config) instead of remembering arbitrary commands, but I don't know where Git stores the file references that get passed to "git update-index --assume-unchanged file".
If you know, please do tell!
It says where in the command - git update-index
So you can't really be editing the index as it is not a text file.
Also, to give more detail on what is stored with the git update-index --assume-unchanged command, see the Using “assume unchanged” bit section in the manual
As others said, it's stored in the index, which is located at .git/index.
After some detective work, I found that it is located at the: assume valid bit of each index entry.
Therefore, before understanding what follows, you should first understand the global format of the index, as explained in my other answer.
Next, I will explain how I verified that the "assume valid" bit is the culprit:
empirically
by reading the source
Empirical
Time to hd it up.
Setup:
git init
echo a > b
git add b
Then:
hd .git/index
Gives:
00000000 44 49 52 43 00 00 00 02 00 00 00 01 54 e9 b6 f3 |DIRC........T...|
00000010 2d 4f e1 2f 54 e9 b6 f3 2d 4f e1 2f 00 00 08 05 |-O./T...-O./....|
00000020 00 de 32 ff 00 00 81 a4 00 00 03 e8 00 00 03 e8 |..2.............|
00000030 00 00 00 00 e6 9d e2 9b b2 d1 d6 43 4b 8b 29 ae |...........CK.).|
00000040 77 5a d8 c2 e4 8c 53 91 00 01 62 00 c9 a2 4b c1 |wZ....S...b...K.|
00000050 23 00 1e 32 53 3c 51 5d d5 cb 1a b4 43 18 ad 8c |#..2S<Q]....C...|
00000060
Now:
git update-index --assume-unchanged b
hd .git/index
Gives:
00000000 44 49 52 43 00 00 00 02 00 00 00 01 54 e9 b6 f3 |DIRC........T...|
00000010 2d 4f e1 2f 54 e9 b6 f3 2d 4f e1 2f 00 00 08 05 |-O./T...-O./....|
00000020 00 de 32 ff 00 00 81 a4 00 00 03 e8 00 00 03 e8 |..2.............|
00000030 00 00 00 00 e6 9d e2 9b b2 d1 d6 43 4b 8b 29 ae |...........CK.).|
00000040 77 5a d8 c2 e4 8c 53 91 80 01 62 00 17 08 a8 58 |wZ....S...b....X|
00000050 f7 c5 b3 e1 7d 47 ac a2 88 d9 66 c7 5c 2f 74 d7 |....}G....f.\/t.|
00000060
By comparing the two indexes, and looking at the global structure of the index, see that the only differences are:
byte number 0x48 (9th on line 40) changed from 00 to 80. That is our flag, the first bit of the cache entry flags.
the 20 bytes from 0x4C to 0x5F. This is expected since that is a SHA-1 over the entire index.
This has also though me that the SHA-1 of the index entry in bytes from 0x34 to 0x47 does not take into account the flags, since it did not changed between both indexes. This is probably why the flags are placed after the SHA, which only considers what comes before it.
Source code
Now let's see if that is coherent with source code of Git 2.3.
First look at the source of update-index, grep assume-unchanged.
This leads to the following line:
{OPTION_SET_INT, 0, "assume-unchanged", &mark_valid_only, NULL,
N_("mark files as \"not changing\""),
PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, MARK_FLAG},
{OPTION_SET_INT, 0, "no-assume-unchanged", &mark_valid_only, NULL,
N_("clear assumed-unchanged bit"),
PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, UNMARK_FLAG},
so the value is stored at mark_valid_only. Grep it, and find that it is only used at one place:
if (mark_valid_only) {
if (mark_ce_flags(path, CE_VALID, mark_valid_only == MARK_FLAG))
die("Unable to mark file %s", path);
return;
}
CE means Cache Entry.
By quickly inspecting mark_ce_flags, we see that:
if (mark)
active_cache[pos]->ce_flags |= flag;
else
active_cache[pos]->ce_flags &= ~flag;
So the function basically sets or unsets the CE_VALID bit, depending on mark_valid_only, which is a tri-state:
mark: --assume-unchanged
unmark: --no-assume-unchanged
do nothing: the default value 0 of the option set at {OPTION_SET_INT, 0
Next, by grepping under builtin/, we see that no other place sets the value of CE_VALID, so --assume-unchanged must be the only command that sets it.
The flag is however used in many places of the source code, which should be expected as it has many side-effects, and it is used every time like:
ce->ce_flags & CE_VALID
so we conclude that it is part of the ce_flags field of struct cache_entry.
The index is specified at cache.h because one of its functions is to be a cache for creating commits faster.
By looking at the definition of CE_VALID under cache.h and surrounding lines we have:
#define CE_STAGEMASK (0x3000)
#define CE_EXTENDED (0x4000)
#define CE_VALID (0x8000)
#define CE_STAGESHIFT 12
So we conclude that it is the very first bit of that integer (0x8000), just next to the CE_EXTENDED, which is coherent with my earlier experiment.
I have a .mid file - this one specifically. Appart from the header chunk here is the relevant part of the midi. The first track chunk contains only meta events and is described as
4D 54 72 6B 00 00 00 52 // Track chunk #1 info
00 FF 58 04 04 02 18 08 // Meta event
00 FF 59 02 00 00 // Meta event
00 FF 51 03 15 CC 5B // Meta event
81 88 70 FF 51 03 16 E3 60 // Meta event
81 70 FF 51 03 1A 28 6E // Meta event
81 70 FF 51 03 1C 9C 38 // Meta event
81 70 FF 51 03 1E 84 80 // Meta event
81 70 FF 51 03 20 B2 89 // Meta event
81 70 FF 51 03 23 36 6C // Meta event
81 70 FF 51 03 2B 98 B7 // Meta event
00 FF 2F 00 // Meta event. End of track #1
This is the second track chunk, wich contains only Note on events:
4D 54 72 6B 00 00 04 02 // Track chunk #2 info
00 FF 21 01 00 // Meta event
00 FF 03 1F 53 69 6E 66 6F 6E 69
61 20 66 72 6F 6D 20 4B 61 6E 74 61 74 61 20 23
31 30 36 20 2D 4A 53 20 // Meta event
00 C0 4A // Midi event
00 B0 07 7C // Midi event
00 B0 0A 52 // Midi event
00 B0 5B 4B // Midi event
96 40 90 4B 50 // Midi "Note on" event
81 63 4B 00 // Midi "Note on" event because of running status
... // No note offs event in this track chunk.
FF 2F 00 // Meta event. End of track #2
This is track #3:
4D 54 72 6B 00 00 03 27 // Track chunk #3 info
... // Some meta and non-note-on, non-note-off midi events
96 40 91 4B 50 // "Note on" event
81 63 4B 00 // Another "Note on" event
... // No note offs
FF 2F 00 // End of track #3
Track chunk #4, #5 and #6 have the same structure as #3; #7 contains two meta events only.
Note that every track chunk which doesn't contain only meta events starts with two Note on events, and no one of them contains a single Note off event.
But this is what synthesia (a midi player) sees when loading the file, just at the start of the song:
The structure of this midi would be a Note on event followed by a Note off event in the same track chunk, but this doesn't happen.
What's happening here?
96 40
90 4B 50 // Midi "Note on" event
81 63
4B 00 // Midi "Note on" event because of running status
^^
The MIDI specification says:
MIDI provides two roughly equivalent means of turning off a note (voice). A note may be turned off either by sending a Note-Off message for the same note number and channel, or by sending a Note-On message for that note and channel with a velocity value of zero. The advantage to using "Note-On at zero velocity" is that it can avoid sending additional status bytes when Running Status is employed.
Due to this efficiency, sending Note-On messages with velocity values of zero is the most commonly used method.
I have SCL011 Card Reader and need to read/write Mifare 1k cards. But I just can't get over Authentification step....
Card Reader should handle Mifare 1k cards:
Antenna ISO/IEC 14443 compliant design
Baudrate up to 848 Kbps
Supported standards:
ISO/IEC 14443-4 Typ A & B
MIFARE: Classic 1K and 4K, DESFire, Ultralight, MIFARE Plus
FeliCa™
NFC forum tag type 1, 2, 3, 4
iCLASS UID*
I have also updated to the latest firmware (1.20)
http://support.identive-group.com/dfu_fw.php?OS=windows&readerno=85
card is connected and I can read the UID of the card with ff ca 00 00 00
I have also tried to read the sector directly without authorization ff b0 00 00 10 and I get message:
69 82 : Command not allowed. Security status not satisfied.
it means I need authorize myself, but if I try ff 82 00 00 06 ff ff ff ff ff ff or any other standard keys I always get back:
69 88 : Command not allowed. SM data objects incorrect.
funny thing is, that I can read and write this card without problems with my Nexus and Lumia phones...
What I'm doing wrong? Thanks for any help!
keys I have already tried:
* ff 82 00 00 06 ff ff ff ff ff ff
* ff 82 00 00 06 a0 b0 c0 d0 e0 f0
* ff 82 00 00 06 a1 b1 c1 d1 e1 f1
* ff 82 00 00 06 a0 a1 a2 a3 a4 a5
* ff 82 00 00 06 b0 b1 b2 b3 b4 b5
* ff 82 00 00 06 4d 3a 99 c3 51 dd
* ff 82 00 00 06 1a 98 2c 7e 45 9a
* ff 82 00 00 06 00 00 00 00 00 00
* ff 82 00 00 06 d3 f7 d3 f7 d3 f7
* ff 82 00 00 06 aa bb cc dd ee ff
Solution: Please google/search "Multiprotocol contactless mobile reader, Reference manual" or "SCL01X Multiprotocol contactless stationary reader".
It is a very nice references to start with SCL reader's APDUs. There are some examples inside.
Answer: In your case P2 value in the APDU Command incorrect and you got SW1SW2 = 0x6988 - "Key number not valid".
Where P2 can have the following values (please refer to MIFARE documentation from NXP for
further details on what is key A and Key B):
• 0x60 to use the Key A
• 0x61 to use the Key B
I write the following bytes into a file named disk.img
FA 8D 36 1B 7C E8 01 00 F4 AC 3C 00 74 0C B4 0E
BB 07 00 B9 01 00 CD 10 EB EF C3 4D 61 79 20 74
68 65 20 66 6F 72 63 65 20 62 65 20 77 69 74 68
20 79 6F 75 21 0D 0A 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
..enough zero to make the size of file 512bytes.
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
The above bytes are proper instructions and magic number that should work when loading into the boot sector. But after I executed "qemu-X86_64 disk.img", error happens.
Error -13 while loading disk.img
Does anyone know how to solve the problem or what is the reason that might lead to this error?
Thank you!
I don't know if you can fill an image with just anything and expect it to work just because you have 55 AA in the correct place. Since you seem to be writing a bootloader make sure your code thinks it is executing at the correct place. It should be in offset 0x7C00 (if I remember this correctly, double check that). You set it by writing the line [org 0x7C00] at the top of your assembly file.
Also I'm not sure you can have only a 512 byte file. Try to make the disk image bigger than that using something like dd if=/dev/zero of=disk.img bs=512 count=2000 and then just copy your bootloader to the first part of the disk using dd again.
Also, you should use the -hda or -fda tags, so it would be qemu -hda disk.img. -hda means hard drive image, and -fda means floppy disk image.