May i know how do i merge my mongodb data files?
The following is my current situation:
At /data/db:
drwxr-xr-x 3 root root 4096 2011-03-15 06:38 .
drwxr-xr-x 3 root root 4096 2011-03-15 00:35 ..
-rwxr-xr-x 1 root root 5 2011-03-15 06:54 mongod.lock
drwxr-xr-x 2 root root 4096 2011-03-15 00:43 social_scraper
-rw------- 1 root root 67108864 2011-03-15 06:39 social_scraper.0
-rw------- 1 root root 134217728 2011-03-15 06:38 social_scraper.1
-rw------- 1 root root 16777216 2011-03-15 06:39 social_scraper.ns
At /var/lib/mongodb :
drwxr-xr-x 5 mongodb mongodb 4096 2011-03-15 07:06 .
drwxr-xr-x 46 root root 4096 2011-03-10 09:40 ..
-rwxr-xr-x 1 root root 0 2011-03-15 07:06 mongod.lock
drwxr-xr-x 2 root root 4096 2011-03-15 07:06 social_scraper
-rw------- 1 root root 67108864 2011-03-15 07:06 social_scraper.0
-rw------- 1 root root 134217728 2011-03-15 07:06 social_scraper.1
-rw------- 1 root root 16777216 2011-03-15 07:06 social_scraper.ns
How do i merge all of the records into /data/db or /var/lib/mongodb ?
The above records ( in both directories ) have been restored using mongod --repair
It will be great if specific steps can be given as I am quite a newbie when it comes to MongoDB maintainence.
I'm using ubuntu 10.04 server edition, MongoDB v1.6.5.
Best Regards.
You can use mongodump to move files from first database in bson files, and than use mongoimport to import dumped files into another database:
mongodump.exe --host localhost:27020 --db dbName
mongorestore.exe --host localhost:27020 --db secondDBName folderWithBsonFiles
Related
I've my application log file which keeps rotating and it has a pattern like below
postgresql-yyyy-mm-dd_hhmmss.log
-rw-------. 1 root root 331 Jun 1 22:04 postgresql-2022-06-01_220333.log.gz
-rw-------. 1 root root 417 Jun 1 22:04 postgresql-2022-06-01_220430.log.gz
-rw-------. 1 root root 289 Jun 2 19:28 postgresql-2022-06-02_000000.log.gz
-rw-------. 1 root root 26802 Jun 2 19:50 postgresql-2022-06-02_192809.log.gz
-rw-------. 1 root root 8440 Jun 2 23:57 postgresql-2022-06-02_195044.log.gz
-rw-------. 1 root root 15016 Jun 3 11:22 postgresql-2022-06-03_000000.log.gz
-rw-------. 1 root root 291 Jun 3 11:24 postgresql-2022-06-03_112405.log.gz
-rw-------. 1 root root 336 Jun 3 11:25 postgresql-2022-06-03_112553.log.gz
-rw-------. 1 root root 397 Jun 3 11:27 postgresql-2022-06-03_112714.log.gz
-rw-------. 1 root root 358 Jun 3 11:29 postgresql-2022-06-03_112901.log.gz
-rw-------. 1 root root 493 Jun 3 11:30 postgresql-2022-06-03_113031.log.gz
-rw-------. 1 root root 418 Jun 3 11:34 postgresql-2022-06-03_113354.log.gz
-rw-------. 1 root root 419 Jun 3 11:39 postgresql-2022-06-03_113920.log.gz
-rw-------. 1 root root 416 Jun 3 11:44 postgresql-2022-06-03_114437.log.gz
-rw-------. 1 root root 417 Jun 3 11:49 postgresql-2022-06-03_114943.log.gz
-rw-------. 1 root root 419 Jun 3 11:55 postgresql-2022-06-03_115530.log.gz
-rw-------. 1 root root 16961 Jun 3 23:56 postgresql-2022-06-03_120047.log.gz
-rw-------. 1 root root 35470 Jun 4 23:56 postgresql-2022-06-04_000000.log.gz
-rw-------. 1 root root 406059 Jun 5 17:56 postgresql-2022-06-05_000000.log
I want to run a small script(unix/linux) such that it creates a symlink in this folder which always points to the latest(current) logfile automatically for e.g
postgres.LOG -> postgresql-2022-06-05_000000.log
This way I don't have to ls and find out what is the current log file before opening, instead just open postgres.LOG.
All rotated logfiles seem gzipped so the glob postgresql-*.log will expand to the logfile of interest:
ln -sf postgresql-*.log postgres.LOG
If you want the symbolic link to be changed automatically each time a new log file is created you need inotify.
Something like this
#! /bin/bash
log_dir=/your/log/dir
inotifywait -m -e create "$log_dir" | while read -r _dir event file
do
if [[ ! -h "$log_dir/$file" && "$file" =~ \.log$ ]]
then
ln -sf "$log_dir/$file" "$log_dir/postgres.LOG"
fi
done
The format of the filename makes sure that the newest file sorts as last using ls. So this should work:
ln -sf $(ls postgresql-*.log | tail -1) postgres.LOG
I'm trying to make a Docker container for a perl script, CooVar, but seem to be running into issues because the tar command is not available.
Dockerfile
FROM perl:5.20
RUN mkdir /tmp/install
WORKDIR /tmp/install
# Install cpan modules
RUN cpanm install Cwd Getopt::Long POSIX File::Basename List::Util Bio::DB::Fasta Bio::Seq Bio::SeqUtils Bio::SeqIO Set::IntervalTree Set::IntSpan
# Download CooVar-v0.07
RUN wget -O /usr/local/bin/CooVar-0.07.tar.gz http://genome.sfu.ca/projects/coovar/CooVar-0.07.tar.gz
RUN tar xvf /usr/local/bin/CooVar-0.07.tar.gz
# Set WORKDIR to /data -- predefined mount location.
RUN mkdir /data
WORKDIR /data
# And clean up
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/install
# Set Entrypoint
ENTRYPOINT ["perl", "/usr/local/bin/coovar-0.07/coovar.pl"]
Note: I think the use of an Entrypoint here is preventing me from exploring a Docker containers file system via docker exec -t -i mycontainer /bin/bash but that's not a big issue as i think I know the problem from the error below.
Error
When I run this from Snakemake I was including ls /usr/bin so I can see what was in the container (very new to Dockerand not sure the best way to develop/debug yet). You can see that the.tar.gzwas not unzipped so it looks liketar` is not working in this case.
drwxrwxr-x 2 root root 2137 Oct 10 12:36 .
drwxrwxr-x 10 root root 147 May 5 2016 ..
-rw-r--r-- 1 root root 176772 Nov 20 2012 CooVar-0.07.tar.gz
-rwxr-xr-x 1 root root 112680 May 5 2016 a2p
-rwxr-xr-x 1 root root 2861 Oct 10 12:35 bp_aacomp
-rwxr-xr-x 1 root root 3696 Oct 10 12:35 bp_bioflat_index
-rwxr-xr-x 1 root root 1605 Oct 10 12:35 bp_biogetseq
-rwxr-xr-x 1 root root 2803 Oct 10 12:35 bp_dbsplit
-rwxr-xr-x 1 root root 2866 Oct 10 12:35 bp_extract_feature_seq
-rwxr-xr-x 1 root root 4002 Oct 10 12:35 bp_fastam9_to_table
-rwxr-xr-x 1 root root 7385 Oct 10 12:35 bp_fetch
-rwxr-xr-x 1 root root 1809 Oct 10 12:35 bp_filter_search
-rwxr-xr-x 1 root root 15121 Oct 10 12:35 bp_find-blast-matches
-rwxr-xr-x 1 root root 3427 Oct 10 12:35 bp_gccalc
-rwxr-xr-x 1 root root 79742 Oct 10 12:35 bp_genbank2gff3
-rwxr-xr-x 1 root root 4617 Oct 10 12:35 bp_index
-rwxr-xr-x 1 root root 1877 Oct 10 12:35 bp_local_taxonomydb_query
-rwxr-xr-x 1 root root 3239 Oct 10 12:35 bp_make_mrna_protein
-rwxr-xr-x 1 root root 4898 Oct 10 12:35 bp_mask_by_search
-rwxr-xr-x 1 root root 3041 Oct 10 12:35 bp_mrtrans
-rwxr-xr-x 1 root root 4292 Oct 10 12:35 bp_mutate
-rwxr-xr-x 1 root root 927 Oct 10 12:35 bp_nexus2nh
-rwxr-xr-x 1 root root 3533 Oct 10 12:35 bp_nrdb
-rwxr-xr-x 1 root root 6331 Oct 10 12:35 bp_oligo_count
-rwxr-xr-x 1 root root 7317 Oct 10 12:35 bp_process_gadfly
-rwxr-xr-x 1 root root 3752 Oct 10 12:35 bp_process_sgd
-rwxr-xr-x 1 root root 24416 Oct 10 12:35 bp_revtrans-motif
-rwxr-xr-x 1 root root 4190 Oct 10 12:35 bp_search2alnblocks
-rwxr-xr-x 1 root root 12519 Oct 10 12:35 bp_search2gff
-rwxr-xr-x 1 root root 2455 Oct 10 12:35 bp_search2table
-rwxr-xr-x 1 root root 2739 Oct 10 12:35 bp_search2tribe
-rwxr-xr-x 1 root root 1779 Oct 10 12:35 bp_seq_length
-rwxr-xr-x 1 root root 2289 Oct 10 12:35 bp_seqconvert
-rwxr-xr-x 1 root root 3833 Oct 10 12:35 bp_seqcut
-rwxr-xr-x 1 root root 4595 Oct 10 12:35 bp_seqpart
-rwxr-xr-x 1 root root 2339 Oct 10 12:35 bp_seqret
-rwxr-xr-x 1 root root 1549 Oct 10 12:35 bp_seqretsplit
-rwxr-xr-x 1 root root 4291 Oct 10 12:35 bp_split_seq
-rwxr-xr-x 1 root root 5912 Oct 10 12:35 bp_sreformat
-rwxr-xr-x 1 root root 3243 Oct 10 12:35 bp_taxid4species
-rwxr-xr-x 1 root root 2940 Oct 10 12:35 bp_taxonomy2tree
-rwxr-xr-x 1 root root 1624 Oct 10 12:35 bp_translate_seq
-rwxr-xr-x 1 root root 1488 Oct 10 12:35 bp_tree2pag
-rwxr-xr-x 1 root root 7504 Oct 10 12:35 bp_unflatten_seq
-rwxr-xr-x 2 root root 36619 May 5 2016 c2ph
-rwxr-xr-x 1 root root 7144 Oct 10 12:33 config_data
-rwxr-xr-x 1 root root 12804 May 5 2016 corelist
-rwxr-xr-x 1 root root 5873 May 5 2016 cpan
-rwxr-xr-x 1 root root 303998 May 5 2016 cpanm
-rwxr-xr-x 1 root root 41760 Oct 10 12:24 enc2xs
Anyone know how I can fix this Dockerfile?
I have installed local::lib from cpan. But the manually one more again to make bootstrap - perl Makefile.PL −−bootstrap. And although I have already a file lib.pm here /usr/share/perl5/local/lib.pm, it when trying to bootstrap it (with the command mentioned above) this error rises:
Loading internal logger. Log::Log4perl recommended for better logging
Attempting to create directory /home/shepherd/perl5
Unable to create /home/shepherd/perl5/lib/perl5/5.28.0: No such file or directory at /usr/share/perl5/local/lib.pm line 717.
So if I have installed the module, and have it may dir, why the bootstrap cannot create /home/shepherd/perl5/lib/perl5/5.28.0 ?
I have and own this files:
ls -l ~/perl5
drwxrwxr-x 2 root root 4096 Nov 18 23:03 bin
drwxrwxr-x 3 root root 4096 Oct 28 18:52 lib
drwxrwxr-x 4 root root 4096 Oct 28 18:52 man
ls -l ~/perl5/lib
drwxrwxr-x 25 root root 4096 Nov 21 08:56 perl5
ls -l ~perl5/lib/perl5
drwxrwxr-x 2 root root 4096 Nov 20 19:54 B
drwxrwxr-x 3 root root 4096 Nov 21 08:56 Business
drwxrwxr-x 3 root root 4096 Oct 28 19:08 Class
drwxrwxr-x 2 root root 4096 Nov 20 19:54 Config
drwxrwxr-x 3 root root 4096 Oct 28 19:09 Devel
drwxrwxr-x 2 root root 4096 Oct 28 19:08 Dist
drwxrwxr-x 3 root root 4096 Nov 20 19:54 File
drwxrwxr-x 2 root root 4096 Nov 20 19:54 Hook
-r--r--r-- 1 root root 42537 Feb 18 2018 Importer.pm
drwxrwxr-x 3 root root 4096 Oct 28 19:10 IPC
drwxrwxr-x 3 root root 4096 Nov 18 23:03 Log
drwxrwxr-x 3 root root 4096 Nov 20 20:48 Mac
drwxrwxr-x 3 root root 4096 Oct 28 19:09 MIME
drwxrwxr-x 3 root root 4096 Oct 28 18:52 Module
-r--r--r-- 1 root root 967 Sep 6 16:40 ok.pm
drwxrwxr-x 2 root root 4096 Nov 19 23:13 OLE
drwxrwxr-x 3 root root 4096 Oct 28 19:08 Perl
drwxrwxr-x 3 root root 4096 Oct 28 19:09 POD2
drwxrwxr-x 2 root root 4096 Oct 28 19:09 Scope
drwxrwxr-x 2 root root 4096 Nov 20 19:56 String
drwxrwxr-x 2 root root 4096 Nov 20 19:54 Task
drwxrwxr-x 3 root root 4096 Oct 28 19:09 Term
drwxrwxr-x 9 root root 4096 Nov 20 19:54 Test
drwxrwxr-x 10 root root 4096 Oct 28 19:09 Test2
-r--r--r-- 1 root root 6393 Sep 6 16:40 Test2.pm
drwxrwxr-x 7 root root 4096 Nov 20 19:55 x86_64-linux-gnu-thread-multi
local::lib misreports the error reason. In this particular case, the problem is that the user running the program (shepherd) doesn't have permission to write to /home/shepherd/perl5/lib/perl5 in order to create /home/shepherd/perl5/lib/perl5/5.28.0.
The user will need to be granted the ability to write to that directory. This should probably be done by changing the ownership of the files from root to shepherd.
chown -R shepherd:shepherd ~/perl5
I downloaded scala,unpacked it and copied to /usr/local/share
After that,I edited ~/.bashrc
export SCALA_HOME=/usr/local/share/scala-2.12.8
export PATH=$PATH:$SCALA_HOME/bin
Anyway,it does not work.
scala -version
Command 'scala' not found, but can be installed with:
sudo apt install scala
I could go for apt install but I don't get it what is wrong.
/usr/local/share$ ll
total 48
drwxr-xr-x 12 root root 4096 јул 1 06:03 ./
drwxr-xr-x 14 root root 4096 јун 23 18:04 ../
drwxr-xr-x 2 root root 4096 апр 21 08:04 appdata/
drwxr-xr-x 2 root root 4096 јун 25 21:03 applications/
drwxr-xr-x 2 root root 4096 јул 25 2018 ca-certificates/
drwxrwsr-x 3 root staff 4096 јул 25 2018 emacs/
drwxrwsr-x 2 root staff 4096 јул 25 2018 fonts/
drwxr-xr-x 3 root root 4096 апр 19 08:44 lua/
drwxr-xr-x 2 root root 4096 јул 25 2018 man/
drwxrwxr-x 6 miki miki 4096 дец 4 2018 scala-2.12.8/
drwxrwsr-x 7 root staff 4096 мај 14 15:05 sgml/
drwxrwsr-x 6 root staff 4096 мај 14 15:05 xml/
Editing .bashrc won't affect an already open shell. You could explicitly source it (source ~/.bashrc) to have it take effect, or open a new shell to reload it.
I need to build mongodb from source to get SSL support.
I ran the following set of commands as per various sources I could find:
sudo git clone git://github.com/mongodb/mongo.git
cd mongo/
git checkout r2.6.0
sudo git checkout r2.6.0
sudo scons --ssl all
This builds without major issues (although it takes a long time) but the output files are huge:
-rwxr-xr-x 1 root root 361M May 2 12:28 bsondump
-rwxr-xr-x 1 root root 148M May 1 15:07 mongo
-rwxr-xr-x 1 root root 359M May 2 12:54 mongobridge
-rwxr-xr-x 1 root root 365M May 2 10:30 mongod
-rwxr-xr-x 1 root root 362M May 2 11:21 mongodump
-rwxr-xr-x 1 root root 361M May 2 11:40 mongoexport
-rwxr-xr-x 1 root root 362M May 2 12:21 mongofiles
-rwxr-xr-x 1 root root 361M May 2 11:48 mongoimport
-rwxr-xr-x 1 root root 361M May 2 12:13 mongooplog
-rwxr-xr-x 1 root root 358M May 2 12:35 mongoperf
-rwxr-xr-x 1 root root 362M May 2 11:30 mongorestore
-rwxr-xr-x 1 root root 248M May 2 11:07 mongos
-rwxr-xr-x 1 root root 362M May 2 11:57 mongostat
-rwxr-xr-x 1 root root 361M May 2 12:04 mongotop
-rwxr-xr-x 1 root root 361M May 2 12:46 perftest
-rwxr-xr-x 1 root root 455M May 2 14:54 test
Am I missing something here?
The raw build produces binaries that contain the debug symbols and you can get rid of them by using the strip command like this:
strip mongod