I want to fetch particular portion from text file using spark scala
Is there any inbuilt function?
If we can do it by using regx then how to do that?
from below line data starts
/bin/rm: cannot unlink `/fabos/link_sbin/lscfg_test': Permission denied
Non-VF
======================
Date:
Mon Jul 8 08:48:40 CEST 2019
Time Zone:
Europe/Berlin
Version:
Kernel: 2.6.14.2
Fabric OS: v7.4.2a
Made on: Thu Jun 29 17:22:14 2017
Flash: Tue Oct 10 09:27:26 2017
BootProm: 1.0.11
supportshow groups enabled:
Unknown key pm:0
os enabled
exception enabled
port enabled
fabric enabled
services enabled
security enabled
network enabled
portlog enabled
system enabled
extend disabled
filter disabled
ficon disabled
iswitch enabled
asic_db enabled
fcip disabled (not applicable to this platform)
ag enabled
dce_hsl enabled
Begin start_port_log_cmd group
Mon Jul 8 08:48:44 CEST 2019
portlogdump:
portlogdump :
time task event port cmd args
-------------------------------------------------
Mon Jul 8 03:27:51 2019
03:27:51.199 FCPH seq 13 28 00300000,00000000,00000591,00020182,00000000
03:27:51.199 PORT Rx 11 0 c0fffffd,00fffffd,0ed10335,00000001
03:27:51.200 PORT Tx 13 40 02fffffd,00fffffd,0ed3ffff,14000000
03:27:51.200 PORT Rx 13 0 c0fffffd,00fffffd,0ed329ae,00000001
03:27:59.377 PORT Rx 15 40 02fffffd,00fffffd,0336ffff,14000000
03:27:59.377 PORT Tx 15 0 c0fffffd,00fffffd,03360ed2,00000001
03:27:59.377 FCPH read 15 40 02fffffd,00fffffd,d0000000,00000000,03360ed2
03:27:59.377 FCPH seq 15 28 22380000,03360ed2,0000052b,0000001c,00000000
03:28:00.468 PORT Rx 13 40 02fffffd,00fffffd,29afffff,14000000
03:28:00.468 PORT Tx 13 0 c0fffffd,00fffffd,29af0ed5,00000001
03:28:00.469 FCPH read 13 40 02fffffd,00fffffd,66000000,00000000,29af0ed5
03:28:00.469 FCPH seq 13 28 22380000,29af0ed5,0000052b,0000001c,00000000
03:28:01.197 FCPH write 15 40 00fffffd,00fffffd,00000000,00000000,00000000
03:28:01.197 FCPH seq 15 28 00300000,00000000,00000591,00020182,00000000
03:28:01.197 PORT Tx 15 40 02fffffd,00fffffd,0ed4ffff,14000000
03:28:01.198 PORT Rx 15 0 c0fffffd,00fffffd,0ed40338,00000001
03:28:09.380 PORT Rx 11 40 02fffffd,00fffffd,033affff,14000000
03:28:09.380 PORT Tx 11 0 c0fffffd,00fffffd,033a0ed6,00000001
03:28:09.380 FCPH read 11 40 02fffffd,00fffffd,d5000000,00000000,033a0ed6
03:28:09.380 FCPH seq 11 28 22380000,033a0ed6,0000052b,0000001c,00000000
Expected Output is like below.. I want data from particular line that can be anything(here from time onwords)
+------------+----+-----+----+---+----------------------------------------
|time |task|event|port|cmd|args
+------------+----+-----+----+---+----------------------------------------
|03:27:51.199|PORT|Rx |11 |0 |c0fffffd,00fffffd,0ed10335,00000001 |
|03:27:51.200|PORT|Tx |13 |40 |02fffffd,00fffffd,0ed3ffff,14000000 |
|03:27:51.200|PORT|Rx |13 |0 |c0fffffd,00fffffd,0ed329ae,00000001 |
|03:27:59.377|PORT|Rx |15 |40 |02fffffd,00fffffd,0336ffff,14000000 |
|03:27:59.377|PORT|Tx |15 |0 |c0fffffd,00fffffd,03360ed2,00000001 |
This is sample data I want data from line starts with
time task event port cmd args
scala> val serverData = spark.read.textFile("serverData.txt")
//get column names from raw data
scala> val schemaArr = serverData.filter(x=>x.matches("\\btime.*\\b")).collect.mkString.split("\\s+").toList
//get required data to transform
scala> val reqData = serverData.filter(x=>x.matches("^\\d.*"))
val df = reqData.map(x=>{
val cols = x.split("\\s+")
(cols(0),cols(1),cols(2),cols(3),cols(4),cols(5))
}).toDF(schemaArr:_*)
please try above regex to extract your data and go on. Hope this will help you!
Can anyone tell me or point me to a section of the specification(s) that clearly demonstrates how from an elementary stream with a series of NALUs how these should be written into a ISO BMFF mdat?
I can see looking at samples and other code that I should have something like: AUD, SPS, PPS, SEI, VideoSlice, AUD etc etc
Things that are not entirely clear to me:
If the SPS and PPS are also stored out of band in the AVCC are they required in the mdat?
If they are required in the mdat when/where should they be written? e.g. just prior to an IDR?
What is the requirement for AUDs?
If I am generating sample sizes for the trun is the calcuation for this? In the example I am working to recreate the first sample in the trun has a size of 22817 however if I look at the first sample in the mdat the NALU size prefix is 22678. The value in the trun appears to be the size of all the NALUs + sizes up to and including the first sample (see my example below)
>
1 0016E405 (1500165) - box.Size
2 6D646174 (mdat) - box.Type
3 00000002 (2) NAL Size
4 0910 - (2) AUD # 5187
5 00000025 (37)
6 27640020 AC248C0F 0117EF01 10000003 00100000 078E2800 0F424001 E84EF7B8 0F844229 C0 (37) # 5193 SPS
7 00000004 (4)
8 28DEBCB0 (4) PPS
9 0000000B (11)
10 06000781 36288029 67C080 (? SEI ?)
11 0000000C (12)
12 06010700 00F00000 03020480 (? SEI is type 6)
13 0000002D (45) # 5269
14 060429B5 00314741 393403CA FFFC8080 FA0000FA 0000FA00 00FA0000 FA0000FA 0000FA00 00FA0000 FA0000FF 80 (SEI ??)
15 00005896 (22678)
16 25888010 02047843 00580010 08410410 0002….. 22678 bytes video # 5322
If the SPS and PPS are also stored out of band in the AVCC are they required in the mdat?
No
If they are required in the mdat when/where should they be written? e.g. just prior to an IDR?
Yes, if you choose to include them, but there is no reason to
What is the requirement for AUDs?
They are optional
If I am generating sample sizes for the trun is the calcuation for this?
The number of bytes in the access unit (AU, aka frame). Which may contain more than one NALU. SPS/PPS/SEI/AUD all counted toward the AU size. The 4 byte size prefixed to each NALUs is also counted in the AU size recored in the trun.
bytes
4 | 3 00000002 (2) NAL Size
2 | 4 0910 - (2) AUD # 5187
4 | 5 00000025 (37)
37 | 6 27640020 AC248C0F 0117EF01 10000003 00100000 078E2800 0F424001 E84EF7B8 0F844229 C0 (37) # 5193 SPS
4 | 7 00000004 (4)
4 | 8 28DEBCB0 (4) PPS
4 | 9 0000000B (11)
11 | 10 06000781 36288029 67C080 (? SEI ?)
4 | 11 0000000C (12)
12 | 12 06010700 00F00000 03020480 (? SEI is type 6)
4 | 13 0000002D (45) # 5269
45 | 14 060429B5 00314741 393403CA FFFC8080 FA0000FA 0000FA00 00FA0000 FA0000FA 0000FA00 00FA0000 FA0000FF 80 (SEI ??)
4 | 15 00005896 (22678)
22678 | 16 25888010 02047843 00580010 08410410 0002….. 22678 bytes video # 5322
------|
22817 | <- bytes total
Starting from El Capitan the system got System Integrity Protection which doesn't allow certain activities for some folders and files. Using Terminal "ls lO" command you can see flags that specific file or folder has.
drwxr-xr-x# 3 root wheel hidden 96 Aug 12 2014 opt
drwxr-xr-x 6 root wheel sunlnk,hidden 192 Nov 28 15:14 private
drwxr-xr-x# 64 root wheel restricted,hidden 2048 Nov 29 13:48 sbin
lrwxr-xr-x# 1 root wheel restricted,hidden 11 Nov 28 15:13 tmp -> private/tmp
drwxr-xr-x# 10 root wheel restricted,hidden 320 Nov 28 15:21 usr
lrwxr-xr-x# 1 root wheel restricted,hidden 11 Nov 28 15:13 var -> private/var
I'm interested in "restricted" flag. How can it be found using Swift without executing terminal commands?
I want to emphasize that executing Process() in code is not an approach that suits the needs.
"Hidden" flag can be detected via "isHidden" property from here URLResourceKey. However there is nothing about "restricted".
Could somebody point me to the right direction?
The things listed by the -O option on the ls tool are the file flags. These can be read by the stat() function in the BSD layer, and are found in the st_flags field of the resulting structure.
The "restricted" flag in ls's output corresponds to SF_RESTRICTED, so you can read it by doing something like this:
func isRestricted(at url: URL) throws -> Bool {
let flags: UInt32 = try url.withUnsafeFileSystemRepresentation { fsRep in
var info = stat()
if stat(fsRep, &info) != 0 {
guard let code = POSIXError.Code(rawValue: errno) else {
throw CocoaError(.fileReadUnknown)
}
throw POSIXError(code)
}
return info.st_flags
}
return flags & UInt32(bitPattern: SF_RESTRICTED) != 0
}
I have some question related the RISC V manual
It has different types of instruction encoding such as R-type,I-type.
Just like the MIPS encoding.
* R-type
31 25 24 20 19 15 14 12 11 7 6 0
+------------+---------+---------+------+---------+-------------+
| funct7 | rs2 | rs1 |funct3| rd | opcode |
+------------+---------+---------+------+---------+-------------+
* I-type
31 20 19 15 14 12 11 7 6 0
+----------------------+---------+------+---------+-------------+
| imm | rs1 |funct3| rd | opcode |
+----------------------+---------+------+---------+-------------+
* S-type
31 25 24 20 19 15 14 12 11 7 6 0
+------------+---------+---------+------+---------+-------------+
| imm | rs2 | rs1 |funct3| imm | opcode |
+------------+---------+---------+------+---------+-------------+
* U-type
31 11 7 6 0
+---------------------------------------+---------+-------------+
| imm | rd | opcode |
+---------------------------------------+---------+-------------+
But it also have something called immediate format:
such as I-immediate, S-immediate and so on
* I-immediate
31 10 5 4 1 0
+-----------------------------------------+-----------+-------+--+
| <-- 31 | 30:25 | 24:21 |20|
+-----------------------------------------+-----------+-------+--+
* S-immediate
31 10 5 4 1 0
+-----------------------------------------+-----------+-------+--+
| <-- 31 | 30:25 | 11:8 |7 |
+-----------------------------------------+-----------+-------+--+
* B-immediate
31 12 11 10 5 4 1 0
+--------------------------------------+--+-----------+-------+--+
| <-- 31 |7 | 30:25 | 11:8 |z |
+--------------------------------------+--+-----------+-------+--+
* U-immediate
31 30 20 19 12 11 0
+--+-------------------+---------------+-------------------------+
|31| 30:20 | 19:12 | <-- z |
+--+-------------------+---------------+-------------------------+
* J-immediate
31 20 19 12 11 10 5 4 1 0
+----------------------+---------------+--+-----------+-------+--+
| <-- 31 | 19:12 |20| 30:25 | 24:21 |z |
+----------------------+---------------+--+-----------+-------+--+
According to the manual, it say those immediate is produced by RISC-V instruction but how are the things related?
What is the point to have immediate format?
The 2nd set of diagrams is showing you how the immediate bits are concatenated and sign-extended into a 32-bit integer (so they can work as a source operand for normal 32-bit ALU instructions like addi which need both their inputs to be the same size).
For I-type instructions it's trivial, just arithmetic right-shift the instruction word by 20 bits, because there's only one immediate field, and it's contiguous at the top of the instruction word.
For S-type immediate instructions, there are two separate fields in the instruction word: [31:25] and [11:7], and this shows you that they're in that order, not [11:7, 31:25] and not with any implicit zeros between them.
B-type immediate instructions apparently put bit 7 in front of [30:25], and the low bit is an implicit zero. (So the resulting number is always even). I assume B-type is for branches.
U-type is also interesting, padding the 20-bit immediate with trailing zeros. It's used for lui to create the upper bits of 32-bit constants (with addi supplying the rest). It's not a coincidence that U-type and I-type together have 32 total immediate bits.
To access static data, lui can create the high part of an address while lw can supply the low part directly, instead of using an addi to create the full address in a register. This is typical for RISC ISAs like MIPS and PowerPC as well (see an example on the Godbolt compiler explorer). But unlike most other RISC ISAs, RISC-V has auipc which adds the U-type immediate to the program counter, for efficient PIC without having to load addresses from a GOT (global offset table). (A recent MIPS revision also added an add-to-PC instruction, but for a long time MIPS was quite bad at PIC).
lui can encode any 4k-aligned address, i.e. a page-start address with 4k pages.
i folowed this link to calibrate touchscreen: http://www.circuitbasics.com/raspberry-pi-touchscreen-calibration-screen-rotation/.
ls -la /dev/input/
total 0
drwxr-xr-x 4 root root 240 Jul 12 18:38 .
drwxr-xr-x 15 root root 3460 Jul 12 18:38 ..
drwxr-xr-x 2 root root 140 Jul 12 18:38 by-id
drwxr-xr-x 2 root root 140 Jul 12 18:38 by-path
crw-rw---- 1 root input 13, 64 Jul 12 18:38 event0
crw-rw---- 1 root input 13, 65 Jul 12 18:38 event1
crw-rw---- 1 root input 13, 66 Jul 12 18:38 event2
crw-rw---- 1 root input 13, 67 Jul 12 18:38 event3
crw-rw---- 1 root input 13, 68 Jul 12 18:38 event4
crw-rw---- 1 root input 13, 63 Jul 12 18:38 mice
crw-rw---- 1 root input 13, 32 Jul 12 18:38 mouse0
crw-rw---- 1 root input 13, 33 Jul 12 18:38 mouse1
root#raspberrypi:/sys/devices/virtual/input# cat input4/uevent
PRODUCT=0/0/0/0
NAME="FT5406 memory based driver"
PROP=2
EV=b
KEY=400 0 0 0 0 0 0 0 0 0 0
ABS=2608000 3
MODALIAS=input:b0000v0000p0000e0000-e0,1,3,k14A,ra0,1,2F,35,36,39,mlsfw
root#raspberrypi:~# cat /etc/ts.conf
# Uncomment if you wish to use the linux input layer event interface
module_raw input
# Uncomment if you're using a Sharp Zaurus SL-5500/SL-5000d
# module_raw collie
# Uncomment if you're using a Sharp Zaurus SL-C700/C750/C760/C860
# module_raw corgi
# Uncomment if you're using a device with a UCB1200/1300/1400 TS interface
# module_raw ucb1x00
# Uncomment if you're using an HP iPaq h3600 or similar
# module_raw h3600
# Uncomment if you're using a Hitachi Webpad
# module_raw mk712
# Uncomment if you're using an IBM Arctic II
# module_raw arctic2
module pthres pmin=1
module variance delta=30
module dejitter delta=100
module linear
I only get response when configuring X with xinput_calibrator. When i enter this command
sudo TSLIB_FBDEVICE=/dev/fb0 TSLIB_TSDEVICE=/dev/input/event1 ts_calibrate
I get optput
xres = 800, yres = 480
selected device is not a touchscreen I understand
Can someone please help me,
Thanks in advance.
I don't have a solution for this, but I believe that it is related to the problem of touches being treated as mouseovers. This bug has been reported several times, but never actually fixed
https://gitlab.gnome.org/GNOME/gtk/-/issues/945
https://bugzilla.gnome.org/show_bug.cgi?id=789041
https://bugs.launchpad.net/ubuntu-mate/+bug/1792787
A bugzilla.gnome.org user named niteshgupta16 created a script that solves this problem, but it was uploaded to pasting/sharing service called hastebin at https://www.hastebin.com/uwuviteyeb.py.
Hastebin deletes files that have not been accessed within 30 days. Since hastebin is a javascript-obfuscated service, this file is not available on archive.org.
I am unable to find an email for niteshgupta16 in order to ask him if he still has uwuviteyeb.py.