I have to do a search in ADN file on SIM card in my STK applet to find someone.
However, if I have to scan the whole number of records in ADN file (even it's a blank record), it will be very slow and I don't know how to get the number of contacts in the ADN file from STK applet.
If anyone of you knows how to get the number of contacts, please share with me and thank you very much in advance.
As the phone has the contact memory usage information (i.e. 34/250), I don't know how the phone get this information and is there any solution that allow STK Applet to get this information from the phone?
Phone select the file and in response to it SIM send the response, for which mobile again run a command which is known as Get Response, In this get response SIM send the details of that file. Now in case of ADN which is a record file command will contains file size and record length so Total records will be Size/Record length.
You can read about these command in more details in 3GPP 11.11
Now regarding searching the data in STK applet is not very time consuming, STK applets are very fast and 250 records are not too much, so in a loop just read record and check for 21 bytes if it is FF then record is empty. Structure of ADM records is:-
1 to X Alpha Identifier O X bytes
X+1 Length of BCD number/SSC contents M 1 byte
X+2 TON and NPI M 1 byte
X+3 to X+12 Dialling Number/SSC String M 10 bytes
X+13 Capability/Configuration Identifier M 1 byte
X+14 Extension1 Record Identifier M 1 byte
In your case length is 34 so first 20 bytes contains the name(here we call it Alpha ID) then number starts. Well I am asking to check 21th byte because it could be a case when user saved a number without any name. So don't check first byte.
Related
I have a lot of tables stored in flat files (in a directory called basepath) and I want to check their number of rows. The best I can so right now is:
c:([] filename:system "ls ",basepath;
tablesize:count each get each hsym `$basepath,/:system "ls ",basepath)
which loads each table entirely into memory and then performs the count (that's quite slow). Is saving as splayed tables the only way to make this faster (because I would only load 1 column and count that) or is there a trick in q that I can use?
Thanks for the help
If you have basepath defined as a string of the path to directory where all your flat tables are stored then you can create a dictionary of the row counts as follows:
q)cnt:{count get hsym x}
q)filename:key hsym `$basepath
q)filename!cnt each filename
t| 2
g| 3
This is where I have flat tables t and g saved in my basepath directory. This stops you from having to use system commands which are often less effiecient.
The function cnt takes the path of each flat table (as a symbol) and returns the number of rows without saving them into memory.
The best solution if you have control of the process of saving such files down is to add an extra step of saving the meta information of the row count somewhere seperate at the same time of saving the raw data. This would allow you to quickly access the table size from this file instead of reading the full tbale in each time.
However, note that to avoid pulling them into memory at all you would have to instead use read1 and look at the headers on the binary data. As you said it would be better to save as a splayed table and read in one column.
UPDATE: I would not recommend doing this and strongly suggest doing the above but for curiosity after looking into using read1 here's an example what what a hacky solution might look like:
f:{
b:read1(y;0;x);
if[not 0x62630b~b[2 4 5];'`$"not a table"];
cc:first first((),"i";(),4)1:b 7+til 4;
if[null ce:first where cc=sums 0x0=11 _ b;:.z.s[x*2;y]];
c:`$"\000" vs "c"$b[11+til ce];
n:first first((),"i";(),4)1:b[(20+ce)+til 4];
:`columns`rows!(c;n);
}[2000]
The q binary file format isn’t documented anywhere, the only way to figure it out is to save different things and see how the bytes change. It’s also subject to changes between versions - the above is written for 3.5 and is probably valid for 3.0-3.5 only, not the latest 3.6 release or anything 2.X.
The given code works in the following way:
reads a chunk from the front of the file
validates that it looks like a flat unkeyed table (flip[98] of a dict[99] with symbol[11] keys)
reads the count of symbols in the list of columns as a little endian 4 byte int
scans through the null terminated strings for that many zero bytes
if the columns are so numerous or verbose that we don’t have them
all in this chunk it will double the size of the chunk and try again
turn the strings into symbols
using the offset we get from the end of the column list, skip a bit
more of the header for the mixed list of columns
then read the count from the header of the first column
Hope this answers your question!
From experimenting with the binary files, it seems that the table count is saved as part of the binary file when you save down a flat file, taking up 4 bytes after the initial object type and column headings which will vary from table to table.
`:test set ([]a:1 2 3;b:4 5 6;c:7 8 9;aa:10 11 12;bb:13 14 15)
q)read1 `:test
0xff016200630b000500000061006200630061610062620000000500000009000300000
0 7 11 31
bytes | example | meaning
---------------------------------------------------------------------------------------
0 - 5 | 0xff016200630b0 | object is a flat table
7 - 11 | 0x05000000 | number of columns (5)
12- 22 | 0x6100620063006161006262 | one byte for the ascii values of column "a" and "b" in hex followed by the one byte separator
23 - 30 | 0x0000050000000900 | 8 bytes that can be skipped
31 - 34 | 0x0300000 | 4 bytes for row count of first column (3)
This should help you understand the function that Fiona posted.
The binary is saved down little-endian meaning the most-significant byte is the right-hand most digit - doing this in decimal for the number 100 would give 001, with the 100's (most significant) on the right and then 10s and finally 1s on the left. In the binary file, each group of 2 digits is a byte.
You can use 1: to read in the contents of a binary file, with additional arguments in the list specifying the offset - where to start reading from, and how many bytes to read. In our case we want to start at byte 31 and read in 4 bytes, specifying the output should be an integer and to cut the input into separate 4 byte chunks.
q)first first (enlist "i";enlist 4)1:(`:test;31;4)
3i
Converting the little-endian bytes into a long gives us the row count. Since this only has to read in 4 bytes instead of the whole file it is a lot quicker.
For a table with 10000 rows and 2 columns there is not much difference:
q)\t 0x0 sv reverse first (enlist "x";enlist 1)1:(`:test10000;31;4)
0
q)\t count get `:test10000
0
For a table with 100m rows and 2 columns:
q)\t 0x0 sv reverse first (enlist "x";enlist 1)1:(`:test10m;31;4)
0
q)\t count get `:test10m
2023
If you have a splayed table instead you can read in the number of elements in one of the columns from bytes 9-13 like so, assuming the column is a simple list:
q)first first (enlist "i";enlist 4)1:(`:a;8;4)
3i
You can read more about reading in from binary files here https://code.kx.com/q/ref/filenumbers/#1-binary-files
You can make what you currently have more efficient by using the following
counttables:{count each get each hsym `$basepath}
This will improve the speed of the count by not including the extra read in of the data as well as the join which you are currently doing. You are correct though that if the tables where saved splayed you would only have to read in the one column making it much more efficient.
If your tables are stored uncompressed there's probably something quite hacky you could do with a read1 on the headers within the file until you find the first column header.
But v hacky :-(
Are you responsible for saving these down? Can you keep a running state as you do?
I have two data files (delimited files) :
- The first one contain 3 columns, ID, num_phone, trafic_etl : the sim card may be 3g, 4g or whatever.
- the second one contain 1 column num_phone_4g : the sim card has to be 4g.
The thing is, I want to fill a oracle table, with numbers with 4g sim card (second file), that has 0 trafic_etl in total, knowing that the first file may have more than one row for same num_phone.
I did do this with sql statement by storing files in tables.
But what I have to do, is using talend for and I am new to this tool.
Thanks in advance.
Images of the two files : File2
File1
Here's a solution using this sample data.
*File 1*
num_phone;trafic_etl;annee;mois;jour
123456;111111;2018;Juillet;20
123457;222222;2018;Juillet;20
123458;0;2018;Juillet;20
123456;333333;2018;Juillet;20
123457;444444;2018;Juillet;20
123458;0;2018;Juillet;20
*File 2*
num_phone_4g
123456
123457
123458
123459
The expected output is 123458 (because it has a total of 0 trafic) and 123459 (because it's not present in file 1; I don't know if this is possible in your use case).
I aggregate the data of file2 by phone number to get the total trafic for each phone number (assuming the date is not important). Then I use this aggregated data as a lookup to file2. In tMap_1, there is a join between the 2 flows on the phone number, and I only output the rows from file2 where the total trafic is null or zero.
Let me know if my assumptions are correct. If they are not, I will update my answer.
I'm currently reading a text book on xv6, and understand this so far ...
Virtual Address: First 20 bits to index into a PTE. The PTE takes these 20 bits and turns them into a Physical Page Number: PPN. The remaining 12 bits are used for offset, which will be the same in both virtual and physical addresses.
Paging: Paging hardware uses first 10 bits of 20 bits in the virtual address to select a page directory entry (PDE). If a PDE is present, uses next 10 bits of virtual address to select a page table entry (PTE). Something like this ...
00 0000 0011 | 00 0000 0010 | 0000 0000 0101
Page Dir. (3) | Page Table E. (2) | Offset (5)
Question: Is the PPN showed in the diagrams the same all across? I also know the difference between a page directory and page table entry is only by 1 bit, which is set to 0 or 1 depending if you are at page directory or table. Is the PPN common between all 3 then? (Physical Address, Page Table, Page Directory).
Hopefully, this answers your question. If you access a 32-bit address, 12-bits are saved for the offset into the page. They play no part in address translation.
The CR3 register points to a page table directory. Although not specified in your diagram, I believe this points to a physical page frame. That page frame contains an array of directories. The top 10 bits in your address are an index into that directories.
So now you have a structure like the one in your diagram. That structure contains a pointer to a physical page frame (PPN) containing a page table. Again this is physical address that would be padded with zeroes. You use the value in the PPN field to find the page table.
Your page table is an array of structures that look just like the directory. What is misleading in your diagram is that the D bit may or may not be set in a page table while it is always clear in a directory. The next 10 bits in your address are an index into this table. Use those to locate the desired page table entry.
As before you have a PPN. On this second iteration, this is a pointer to a physical address BUT now it is the actual memory page you want to access. Pad the 20 bits of the PPN with zero and add the lower 12 bits of your address and you have the physical address.
if i have a Virtual Address: 0xF3557100 , how do i convert it to Physical Address and what are the Values of Offset, Page Directory and Page Table ?
The PTE (page table entry) for that address has the value 0x87124053
thnx
Sadly, what you are asking is system dependent. You would need to know the size of the page to begin with.
In the simplest case, the lowest order bits corresponding to the page size are the offset and the remaining high order bits specify the page table entry.
You say that you have the value of he page table entry. You then need to know the structure of the page table entry. Some part of that will indicate the physical address. Other parts will define page attributes.
In short, we'd need to know a whole lot more information.
In general from this info you can not translate a VA to PA.
Each architecture has some constant value for PAGE_SHIFT. as your address is 32 bit, most of such architecture has 12 bit PAGE_SHIFT value.
this value determines the offset value so your offset value is 12 bits.that also means your page size is 4096 bytes. even though a architecture can support more than one value for PAGE_SHIFT, we take case of 12 bits offset which is usually default value in most systems making page of 4096
PTE contains address of the page frame/number along with other status and protection information.Lower 12 bits in PTE are used for status and protection while other 20 bits are used for PPN. as a principle virtual frame number is mapped to physical frame number and offset is same in both. so exclude lower most 12 bits from PTE and append 12 lower most bits from va.
so offset from va is 0x100 so physical address is 0x87124100
according to 10-10-12 rule (there is no general rule for this division)
offset = 12 bits
page table = page directory=10 bits
now you CAN easily calculate relevant bits value from given address.
1111001101 0101010111 000100000000
page directory offset = 1111001101
page table offset = 0101010111
page offset = 000100000000
I've started to do some programming in ILE RPG and I'm curious about one thing - what exactly is the record format? I know that it has to be defined in physical/logical/display files but what exactly it does? In an old RPG book from 97 I've found that "Each record format defines what is written to or read from the workstation in a single I/O operation"
In other book I have found definition that record format describe the fields within a record(so for example length, type like char or decimal?).
And last, what exactly means that "every record within a physical file must have an identical record layout"?
I'm a bit confused right now. Still not sure what is record format :F.
Still not sure what is record format :F
The F Specification: This specification is also known as the File specification. Here we declare all the files which we will be using in the program. The files might be any of the physical file, logical file, display file or the printer file. Message files are not declared in the F specification.
what exactly means that "every record within a physical file must have an identical record layout"?
Each and every record within one physical file has the same layout.
Let's make a record layout of 40 characters.
----|---10----|---20----|---30----|---40
20150130 DEBIT 00002100
20150130 CREDIT 00012315
The bar with the numbers is not part of the record layout. It's there so we can count columns.
The first field in the record layout is the date in yyyymmdd format. This takes up 8 characters, from position 1 to position 8.
The second field is 2 blank spaces, from position 9 to position 10.
The third field is the debit / credit indicator. It takes up 10 characters, from position 11 to position 20.
The fourth field is the debit / credit amount. It takes up 8 positions, from position 21 to position 28. The format is assumed to be 9(6)V99. In other words, there's an implied decimal point between positions 26 and 27.
The fifth field is more blank spaces, from position 29 to position 40.
Every record in this file has these 5 fields, all defined the same way.
A "record format" is a named structure that is used for device file I/O. It contains descriptions of each column in the 'record' (or 'row'). The specific combination of data types and sizes and the number and order of columns is hashed into a value that is known as the "record format identifier".
A significant purpose is the inclusion by compilers of the "record format identifier" in compiled program objects for use when the related file is opened. The system will compare the format ID from the program object to the current format ID of the file. If the two don't match, the system will notify the program that the file definition has changed since the program was compiled. The program can then know that it is probably going to read data that doesn't match the definitions that it knows. Nearly all such programs are allowed to fail by sending a message that indicates that the format level has changed, i.e., a "level check" failed.
The handling of format IDs is rooted in the original 'native file I/O' that pre-dates facilities such as SQL. It is a part of the integration between DB2 and the various program compilers available on the system.
The 'native' database file system was developed using principles that eventually resulted in SQL. A SQL table should have rows that all hold the same series of column definitions. That's pretty much the same as saying "every record within a physical file must have an identical record layout".
Physical database files can be thought of as being SQL tables. Logical database files can be thought of as being SQL views. As such, all records in a physical file will have the same definitions, but there is some potential variation in logical files.
A record format It's something you learn in old school. You read a file (table) and update/write through a record format.
DSPFD FILE(myTable)
Then you can see everything about the file. The record format name is in there.
New or Young Developers believe that every record in a physical file must be identical, but in ancient times, the dinosaurs walk on earth and in one single file you could have several types of records or "record formats", so as the name indicates a record format is the format of a record within a file.