Environment:
Jib version: 0.22
Build tool: Gradle
OS: Linux
Description of the issue:
We are adding a large sub-directory as a layer (~234MB)
Jib seems to build this layer locally even though it is cached locally and this operation takes around 25 seconds.
2022-09-29 00:28:25,133 sn:] - Jib Builder Log [DEBUG]: Building layer built sha256:a5ca6eb638e9710574d074247f77b640c0b1d86942c5e0fb1deaf3452325a7cb
This process takes about 25 seconds, then it says that layer already exists
2022-09-29 00:28:25,456 sn:] - Jib Builder Log [INFO]: Skipping push; BLOB already exists on target registry : digest: sha256:a5ca6eb638e9710574d074247f77b640c0b1d86942c5e0fb1deaf3452325a7cb, size: 234630578
bash-4.2$ find /tmp/jib_image_cache/ -name a5ca6eb638e9710574d074247f77b640c0b1d86942c5e0fb1deaf3452325a7cb -exec ls -lrt {} \;
total 229132
-rw-r----- 1 tomcat tomcat 234630578 Sep 29 00:28 7161988df87ff3006f5803117549fb6d14109073edfcde7b2d0d549dc706ad94
bash-4.2$ find /tmp/jib_image_cache/ -name a5ca6eb638e9710574d074247f77b640c0b1d86942c5e0fb1deaf3452325a7cb -exec ls -lrtd {} \;
drwx------ 3 tomcat tomcat 90 Sep 29 06:56 /tmp/jib_image_cache/layers/a5ca6eb638e9710574d074247f77b640c0b1d86942c5e0fb1deaf3452325a7cb
Expected behavior:
Is there any way to speed up the process?
There are 680 files in 38 sub directories under main directory.
Is it a good idea to add each sub directory or file as a layer?
Related
We had some 12 agents (vsts-agent-linux-x64-2.188.4) running on one Az VM (Ubuntu 20.04.2 LTS) as processes (./config.sh && screen ./run.sh). All was well..
I had to run some command related to /tmp folder but it kept showing busy and we suspected that our Agents might be using /tmp. Unfortunately instead of any other clean way of stopping the agents, we killed all processes on this VM manually, including the agents'.
After the /tmp related command ran successfully, I tried running screen ./run.sh from one of the agent directories. And I got an error:
Failed to create CoreCLR, HRESULT: 0x80004005
I also had tried :
.agent2/run.sh and I got the error :
ldd: ./bin/libcoreclr.so: No such file or directory
ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
ldd: ./bin/System.Net.Http.Native.so: No such file or directory
Failed to create CoreCLR, HRESULT: 0x80004005
I even downloaded a new .tar for the agent and ran a fresh ./config . But I get the same error on ./config as well
Is there a solution to this? Please help
export COMPlus_EnableDiagnostics = 0, and then running ./config from the agent directory. worked!
I had this issue when running as the non-privileged user specified in the systemd file but running as root user worked fine.
Finally used:
strace -f -o trace.log /<executable path>/<executable name>
Which led me to:
9183 mknod("/tmp/clr-debug-pipe-9183-8112345738-in", S_IFIFO|0700) = -1 EACCES (Permission denied)
This caused me to compare the /tmp directory between working and non-working boxes.
[<not-working-hostname>]$ ll /
drwxrwxr-x 7 root root 93 Jan 5 21:37 tmp
[<working-hostname>]$ ll /
drwxrwxrwt 7 root root 93 Jan 5 21:59 tmp
(Note the r-x vs rwt)
Fix:
[<hostname>]# chmod 1777 /tmp
after days of regular fight with postgis I stuck on this:
postgres=# CREATE EXTENSION postgis; ERROR: could not load library
"/usr/pgpro-9.6/lib/rtpostgis-2.4.so": libgdal.so.20: cannot open
shared object file: No such file or directory
and have no idea how to pass it, because library path is correct...
[combat#urpordfinal ~]$ ls -alt /usr/pgpro-9.6/lib/ total 13080
-rwxr-xr-x 1 root root 1440056 Apr 23 11:52 postgis_topology-2.4.so
drwxr-xr-x 5 root root 8192 Apr 23 11:52 .
-rwxr-xr-x 1 root root 1878728 Apr 23 11:52 rtpostgis-2.4.so
I'm running pgsql 9.6.8 and psotgis build from source
You have to make sure that libgdal.so.20 is on the shared library path.
Find out where the library is and add that directory to the shared library path.
On Linux, you would normally do that by adding the directory to /etc/ld.so.conf (or, better, to a PostGIS configuration file in /etc/ld.so.conf.d) and running ldconfig.
I've got a chrome os app crashing regularly and causing all other chrome processes to crash as well.
I'm seeing crash reports in chrome://crashes, but no way to see the details of the report. I also can't find any minidump files to analyse.
What is the way to get crash report internals in chrome os?
Try the solution in this SO post.
root#localhost:-$ mkdir /tmp/misc && chmod 777 /tmp/misc
root#localhost:-$ cd /tmp
root#localhost:-$ watch -n 1 'find . -mmin -1 -exec cp {} /tmp/misc/ \;'
Then, as a regular user (not root):
google-chrome --enable-logging --v=1
Once you see files created by the watch command, run:
root#localhost:-$ ls -l
-rw------- 1 root root 230432 Apr 16 09:06 chromium-renderer-minidump-2113a256de381bce.dmp
-rw------- 1 root root 230264 Apr 16 09:12 chromium-renderer-minidump-95889ebac3d8ac81.dmp
-rw------- 1 root root 231264 Apr 16 09:13 chromium-renderer-minidump-da0752adcba4e7ca.dmp
-rw------- 1 root root 236246 Apr 16 09:12 chromium-upload-56dc27ccc3570a10
-rw------- 1 root root 237247 Apr 16 09:13 chromium-upload-5cebb028232dd944
Now you can use breakpad to work on the *.dmp files.
You need to be in dev mode in order to access the crash reports. There's no way otherwise to access where the crashes are saved (system crashes under /var/spool/crash and browser/user crashes under /home/chronos/*/crash/).
However, if you're using official Chrome OS builds, we don't currently publish the symbols for the binaries, so it'll probably be a bit difficult to debug using those minidumps.
I followed some online guides trying to get some headless VMs to start/suspend automatically at boot/shutdown on my Mac. I can't get it to work at all. This is my first time trying to get script runs on Startup/Shutdown, so it could be that I'm just missing something very basic and if that's the case I apologize.
These are the steps I followed:
Created a directory /Library/StartupItems/HeadlessVM
Created two files within that directory:
-rwxr--r-- 1 root wheel 242 Feb 19 19:05 HeadlessVM
-rw-r--r-- 1 root wheel 188 Feb 20 12:42 StartupParameters.plist
Contents for HeadlessVM
$ cat HeadlessVM
#!/bin/sh
. /etc/rc.common
StartService ()
{
ConsoleMessage "Starting HeadlessVM"
/usr/local/bin/runvmheadless
}
StopService ()
{
ConsoleMessage "Suspending HeadlessVM"
/usr/local/bin/suspendvmheadless
}
RunService "$1"
Contents for StartupParameters.plist
$ cat StartupParameters.plist
{
Description = "Runs/Suspends Virtual Machine Headless on OS X Startup/Shutdown";
Provides = ("HeadlessVM");
Uses = ("Disks");
OrderPreference = ("Late");
}
Created my script files, that will perform both actions:
-rwxr-xr-x# 1 xxxxxxx admin 164 Feb 19 01:06 runvmheadless
-rwxr-xr-x# 1 xxxxxxx admin 160 Feb 19 01:19 suspendvmheadless
Contents for runvmheadless
$ cat runvmheadless
#!/bin/bash
"/Applications/VMware Fusion.app/Contents/Library/vmrun" -T fusion start "/Volumes/Archive/Virtual Machines/vm.vmwarevm/vm.vmx" nogui
Contents for suspendvmheadless
$ cat suspendvmheadless
#!/bin/bash
"/Applications/VMware Fusion.app/Contents/Library/vmrun" -T fusion suspend "/Volumes/StaticData/Virtual Machines/vm.vmwarevm/vm.vmx"
My troubleshooting so far:
If I run the scripts from the terminal, they work as intended.
If I run sudo /sbin/SystemStarter (start or stop) "HeadlessVM" it also works.
On console I only see the following when I reboot, no clue on what's wrong tho.
2/20/12 12:11:09.128 PM SystemStarter: Runs/Suspends Virtual Machine Headless on OS X Startup/Shutdown (100) did not complete successfully
Appreciate any help, Thank you.
I found what was wrong. The code above is fine, the problem is that my scripts are trying to get data from an encrypted secondary disk which was not available at boot time.
I used this in order to bypass this problem:https://github.com/jridgewell/Unlock
Thanks
When I cap deploy my Symfony2 project, then log into my server I see that the the dev (app_dev.php) runs ok but the prod version (app.php) does not.
The error is
[Tue Jan 03 14:31:48 2012] [error] [client xxx.xxx.xxx.xxx] PHP Fatal error: Uncaught exception 'RuntimeException' with message 'Failed to write cache file "/var/www/example/prod/releases/20120103202539/app/cache/prod/classes.php".' in /var/www/example/prod/releases/20120103202539/app/bootstrap.php.cache:1079\nStack trace:\n#0 /var/www/example/prod/releases/20120103202539/app/bootstrap.php.cache(1017): Symfony\\Component\\ClassLoader\\ClassCollectionLoader::writeCacheFile('/var/www/example/p...', '<?php ????name...')\n#1 /var/www/example/prod/releases/20120103202539/app/bootstrap.php.cache(682): Symfony\\Component\\ClassLoader\\ClassCollectionLoader::load(Array, '/var/www/example/p...', 'classes', false, false, '.php')\n#2 /var/www/example/prod/releases/20120103202539/web/app.php(10): Symfony\\Component\\HttpKernel\\Kernel->loadClassCache()\n#3 {main}\n thrown in /var/www/example/prod/releases/20120103202539/app/bootstrap.php.cache on line 1079
Looking at the recently deployed cache directory I see:
drwxrwxrwx 4 root root 4096 Jan 3 14:28 .
drwxrwxr-x 5 root root 4096 Jan 3 14:28 ..
drwxr-xr-x 6 www-data www-data 4096 Jan 3 14:28 dev
drwxrwxr-x 7 root root 4096 Jan 3 14:28 prod
I can fix the issue with chown -R www-data.www-data prod/ but I wondered if I can stop this from happening in the first place? And why do the directories have different owners?
This happens because your web-server is running by user, who is not able to write to just created cache/prod directory.
There are two solutions, which I know and use. First, add extra commands to run after deployment to Capfile. Capfile will like this:
load 'deploy' if respond_to?(:namespace) # cap2 differentiator
Dir['vendor/bundles/*/*/recipes/*.rb'].each { |bundle| load(bundle) }
load Gem.find_files('symfony2.rb').last.to_s
after "deploy:finalize_update" do
run "sudo chown -R www-data:www-data #{latest_release}/#{cache_path}"
run "sudo chown -R www-data:www-data #{latest_release}/#{log_path}"
run "sudo chmod -R 777 #{latest_release}/#{cache_path}"
end
load 'app/config/deploy'
Second solution is more elegant. You specify correct user, who can write to cache in deploy.rb and make sure that you don't use sudo:
set :user, "anton"
set :use_sudo, false
In the last version of capifony, they've added the option to set writable directories.
Here's the official article which explains what I've written below : http://capifony.org/cookbook/set-permissions.html
You have to deploy using sudo (not a good practice, but it gets the job done)
set :use_sudo, false
# To prompt the sudo password
default_run_options[:pty] = true
and tell capifony which files to make cache and logs folder writable :
set :writable_dirs, ["app/cache", "app/logs"]
set :webserver_user, "www-data"
set :permission_method, :acl
(you have to install acl on your machine, or use :chwon instead of :acl)
EDIT :
I've just realized that this is not enough, the "set_permissions" task is not automatically called, so you have to explicitly run
cap deploy:set_permissions
Or add this line in your deploy.rb :
before "deploy:restart", "deploy:set_permissions"
I solved this problem by adding cache folder to shared folders.
set :shared_children, [app_path + "/cache", app_path + "/logs", web_path + "/uploads", "vendor"]
This way the directory is not recreated each time during deployment, so there is no problem with permissions.
Yes, don't need recreate cache every time after deploy, this solution is logical and pragmatical.
Second solution from Anton - is work if you cache folder permission true in develop environment