r.regression.series is not working for large number of rasters - linear-regression

I have been working with in GRASS GIS for linear regression on large number of rasters. I was using
r.regression.series
but it is only working for only 280 rasters and giving this error
ERROR: Unable to open input file
</home/…./f_format>:
Too many open files
I also change the number in conf. file but its not working
/etc/security/limits.conf
# <domain> <type> <item> <value>
your_username hard nofile 1500

Related

unable to get the distance informations using GROMACS commands (Tutorial file), what should i do now?

I was following the Gromacs MD Simulation tutorial, but an error occured in the 1st command during "Analyzing Protein-Ligand Interactions and Ligand Dynamics".
I can't edit the mentioned files, as these are binary files.
Please suggest something to resolve this issue.
I tried to resolve the error to get the distance information.
This is the command I used :
gmx distance -s md_0_10.tpr -f md_0_10_center.xtc -select 'resname "JZ4" and name OAB plus resid 102 and name OE1' -oall
Error cccurred:
[Program: gmx distance, version 2021.4-Ubuntu-2021.4-2
Source file: src/gromacs/trajectoryanalysis/modules/distance.cpp (line 218)
Function: void gmx::analysismodules::{anonymous}::checkSelections(const SelectionList&)
Inconsistency in user input:
Selection 'resname "JZ4" and name OAB plus resid 102 and name OE1' does not
evaluate into an even number of positions (there are 1 positions)
]
Screenshot of the problem

Relative Path (pathlib) name working on MAC OS but on Windows gives me a error

Currently I am working a project that has have been using the pathlib library so I can work on my Windows desktop when I need too and on my MacBook Pro. Essentially be able to work between both operating systems. I have not have any issues at all until right now. Here is the set up:
I have a pipeline set up to automatically save a .joblib and a whole lot of .png files that will go to a directory called
output_dir = Path('../Trained_Models/Differential_gene_analysis/A Kidney Cancer Transcriptome Molecular Signature Identifies Tumors with Tumor Thrombus/Models train on TCGA data and test on Rodriguez data/Oct-XX-20XX')
For example, if I want to save a .joblib file under the name RandomForest_TumorThrombus_104.joblib,I would use the command
joblib.dump(model ,output_dir / 'RandomForest_TumorThrombus_104.joblib')
On my MacBook Pro, I have no issues when this is ran, but on Windows it gives me the following error
FileNotFoundError: [Errno 2] No such file or directory: '..\\Trained_Models\\Differential_gene_analysis\\A Kidney Cancer Transcriptome Molecular Signature Identifies Tumors with Tumor Thrombus\\Models train on TCGA data and test on Rodriguez data\\Oct-17-2022\\RandomForest_TumorThrombus_104.joblib'
I have tried to use the .resolve() method to get the absolute path but still gives me the same error. I have tried to experiment to try to see what is goin on such as using os.path.exists(). When using the os.path.exists() method I get True for the follwoing command:
os.path.exists(output_dir)
So it does indeed recognize that the directory exists. The next thing I tried was to rename the file to something like dddddd.joblib and that worked. But I find that only a few names for the file would allow me to save the files. During debug I found that the most recent Traceback occurs here:
with open(filename, 'wb') as f:```
I was wondering if anyone here had any idea what was going on here and how I can fix this issue? Please and Thank you.
The solution was to enable long paths on Windows.

Impossible to merge osm.pbf file properly

I've recently started to work on a project with SRTM datas and I've extracted pbf file using phyghtmap.
To start I'm getting hgt files, converting them to tif using the following command : gdal_fillnodata.py data.hgt data.tif
Then I'm warping them with gdalwarp -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m" -r bilinear -tr 90 90 data.tif warp-90.tif
And finally creating the pbf file with phyghtmap --max-nodes-per-tile=0 -s 10 -0 --pbf warp-90.tif
The results is a list of pbf files. They are perfectly fine when I load them into PostGIS with osm2pgsql. But I want to merge them to fasten the import.
I've tried all the major solutions:
osmium merge *.pbf -o merged.pbf
convert pbf to o5m then osmconvert64 *.o5m -o=merge.o5m then convert back to pbf
merging two by two with osmosis --read-pbf lon4.00_5.00lat44.00_45.00_local-source.pbf --read-pbf lon5.00_6.00lat44.00_45.00_local-source.osm.pbf --merge --write-pbf osmo_merge.osm.pbf
None of them worked and the result is only a very small fraction of the data merged in the result file.
Am I doing something wrong?
Note: If I load all pbf with --append it works, but it take ages for a very small portion of the world.
I found the issue. I wasn't setting the --start-node-id and --start-way-id in my script so all my pbf was using the same id range. Now I'm assigning unique ID and it work like a charm :)

What corruption is indicated by WinDbg and !chkimg?

I am having often BSODs and WinDbg report similar corruption for most of them
4: kd> !chkimg -lo 50 -d !nt
fffff80177723e6d-fffff80177723e6e 2 bytes - nt!MiPurgeZeroList+6d
[ 80 fa:00 e9 ]
2 errors : !nt (fffff80177723e6d-fffff80177723e6e)
and
CHKIMG_EXTENSION: !chkimg -lo 50 -d !nt
fffff8021531ae6d-fffff8021531ae6e 2 bytes - nt!MiPurgeZeroList+6d
[ 80 fa:00 aa ]
2 errors : !nt (fffff8021531ae6d-fffff8021531ae6e)
What does it mean? What with what is compared and how it can be that corruption is similar? Does it explicitly indicates RAM problem?
UPDATE
What do these numbers mean? fffff80177723e6d and fffff8021531ae6d? What does it mean, that endings conincide?
What does the following code mean: nt!MiPurgeZeroList+6d?
I already answered this on superuser.com. Windbg downloads the original Exe/DLLs from the Symbol Server and now the chkimg command detects corruption in the images of executable files by comparing them to the copy on a symbol store.
All sections of the file are compared, except for sections that are
discardable, that are writeable, that are not executable, that have
"PAGE" in their name, or that are from INITKDBG. You can change this
behavior can by using the -ss, -as, or -r switches.
!chkimg displays any mismatch between the image and the file as an
image error, with the following exceptions:
Addresses that are occupied by the Import Address Table (IAT) are not checked.
Certain specific addresses in Hal.dll and Ntoskrnl.exe are not checked, because certain changes occur when these sections are loaded.
To check these addresses, include the -nospec option.
If the byte value 0x90 is present in the file, and if the value 0xF0 is present in the corresponding byte of the image (or vice
versa), this situation is considered a match. Typically, the symbol
server holds one version of a binary that exists in both uniprocessor
and multiprocessor versions. On an x86-based processor, the lock
instruction is 0xF0, and this instruction corresponds to a nop (0x90)
instruction in the uniprocessor version. If you want !chkimg to
display this pair as a mismatch, set the -noplock option.
If the RAM is fine, check the HDD / HDD cables for errors (disk diag tool and run chkdsk to detect and fix NTFS issues). You can also connect the HDD to different SATA port on the mainboard.

Listing the volumes on Solaris OS

I am new to solaris OS, and trying to write a script which collects volume data from solaris box.
We did a similar script for Linux, and we used "df -P" command to list the volumes, and select the entries that start with "/dev".
By default, in linux, i could see a volume "/dev/sda1".
when i run df command on solaris box(df -k),i could not see any entry similar to (/dev/*) in my output.
When i mounted a CD, i could see an entry in df output as below.
/dev/dsk/c1t1d0s2 57632 57632 0 100% /media/VBOXADDITIONS_5.0.14_105127
So, in solaris, what is the pattern, i should look for to pick the volumes?
And, why am I not seeing at least one volume in the pattern /dev/
is it "/dev" or something else?
I am using solaris 11 image on oracle virtual box.
When i try "format" command, i could see 3 disks:
AVAILABLE DISK SELECTIONS:
0. c1d0 <VBOX HAR-8ea18e8b-2b2a0a5-0001-31.25GB> testvolu
/pci#0,0/pci-ide#1,1/ide#0/cmdk#0,0
1. c2d0 <VBOX HAR-b4343b55-dbed77c-0001 cyl 1020 alt 2 hd 64 sec 32>
/pci#0,0/pci-ide#1,1/ide#1/cmdk#0,0
2. c3t0d0 <ATA-VBOX HARDDISK-1.0 cyl 1009 alt 2 hd 64 sec 32>
/pci#0,0/pci8086,2829#d/disk#0,0
But, i dont see any partition in "df -k"
Also, i read here(https://docs.oracle.com/cd/E19455-01/805-6331/6j5vgg680/index.html), that disk names should be in "/dev/dsk/*" format.
Solaris 11 uses ZFS which has no one to one relationship between volumes (partitions) and file systems.
You can look at zpool status output to get the underlying devices.
$ zpool status
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
Here, the whole c1t0d0 disk is used, hence no sx or px suffix.