I am trying to push app to cloud foundry from Jenkins. And it complains of this :
org.cloudfoundry.client.v2.ClientV2Exception: CF-AppResourcesFileModeInvalid(160003): The resource file mode is invalid: File mode '444' with path '.git/objects/pack/pack-af4cdbe6faac9d245253dafc1ecae06dc3fa5816.pack' is invalid. Minimum file mode is '0600'
at org.cloudfoundry.util.JobUtils.getError(JobUtils.java:81)
at reactor.core.publisher.MonoThenMap$ThenMapMain.onNext(MonoThenMap.java:120)
at reactor.core.publisher.FluxFilter$FilterSubscriber.onNext(FluxFilter.java:96)
I have tried:
1.Doing chmod 666 ( and even 777) before the build step.
2.Adding these in my .cfignore:
scripts
.git/
.git/objects/pack/*
plugins/**/*
/.bundle
tmp/
.pack
Wiping off workspace in jenkins and app on cf before another try.
Nothing works.
One interesting thing is after a fresh commit to .cfignore (editing a line and pushing to git) , the first build in jenkins works. Subsequent build fails.
Any help?
Thanks!
The root issue is that the Cloud Foundry Java Client pushes the entire content of the configured path to the server. The Cloud Foundry CLI automatically filters out source control directories (and possibly all hidden directories) this filtering out the most common places to see < 0600, but that’s not actually documented anywhere so we don’t match that behavior. I’ve chatted with the lead of the CLI and they’ll document that behavior at which point we’ll implement what they spec.
The .cfignore file does’t work in the client yet either, but once that is properly spec’d by the CLI team, we’d work that issue as well.
Related
I am building a very small Node/Express API app in Azure using Twilio to route communication for a small group. I initially built out a data structure for users in CosmosDB but found out it's minimum $24 per month, which is way over budget for something that will likely hold 20 or so records. Because of this, is seems much more reasonable to just build this into a json file that sits in a ./json subfolder. However, it has occurred to me that whenever I deploy, I would be overwriting this file with the default file I have locally. I have been working via the Azure App Service tool in Visual Studio Code and can't figure out a way to make it ignore the file.
I can go into Kudu and copy the file down each time before I deploy, but I will eventually forget and this sounds like a very brittle process.
I added a json/ line to .gitignore, but that has no effect on the deployment (as expected).
I also added "appService.zipIgnorePattern": ["json{,/**}"] to the settings.json file, but instead of just ignoring that folder on the server, it erases it on deploy (the zip ignores it and then it wipes/replaces the whole wwwsite folder). Looking for the file gives me {"Message":"'D:\\home\\site\\wwwroot\\json\\users.json' not found."}
I was hoping there is a setting that would deploy, replacing all folders in the package, and ignoring all content in the ./json folder. Does this exist?
Alternative solution, 2021:
Instead of excluding folders, select the folder that you do want to deploy. Data in other folders will not be affected.
Deploy from: edit .vscode/settings.json in your local project and add "appService.deploySubpath": "./folderToDeploy"
Deploy to: In the Azure Portal go to your app service. Under Configuration / Application Settings add a new Application Setting with name SCM_TARGET_PATH and value ./folderToDeployTo
Using VS Code right+click deploy will deploy the contents of the folder. I was able to work around this by adding Azure as a remote branch and using .gitignore. I placed my json file inside a random folder (content/json) then placed /content/json in my .gitignore file.
Since today I have the problem that the CF push task in bamboo will hang on the Uploading app files step. I didn't change anything besides some environment variables and of course a bit of code. The log looks like this:
Creating/updating app App-X...
Uploading App-X... Uploading app files
from: /home/bamboo/bamboo-agent-home/xml-data/build-dir/App-X-JOB1/
This will run forever without the Application Cloud getting any updates.
Are there any good ways to debug CF push tasks?
You should try to debug your deployment as close as possible to the way you execute it on bamboo. If you do a cf push, the cf cli automatically tails the staging logs for you, so if you don't see any output during staging it's unlikely that cf logs will tell you anything more.
Since the last thing you're seeing from the log snippet posted above is an "uploading" statement, I would check these things:
use a more recent version of the cf cli, they recently added an upload progress bar which might be useful
check if you have either a .gitignore or .cfignore file in the dir you're pushing from so that you don't push any files you don't need (docs)
hanging pushes may be a plain malfunction of the CF platform you're using, e.g. due to inavailability of CFs internal blob store, no more cells available for staging etc., there's tons of possibilites here
Disclaimer: I'm a co-founder at Meshcloud and we provide public and private (also on-site/hybrid) managed Cloud Foundry hosting in the EU.
You can get the full trace log of a push by using cf push -v, which might give you a hint of where (and possibly why) it is stuck.
Other useful debugging tools are cf logs / cf logs --recent and cf events, but if the push itself simply hangs (rather than failing), they might not produce anything at all. Have you tried waiting for the timeout?
Also, please make sure you're using the latest cf cli version.
I've created a liberty bluemix project. Then bluemix created the GIT project. I've downloaded it in eclipse and now I want to enable more features.
There's a server.xml there
but no matter what features I add there, bluemix logs says I am still using the default ones.
I am just pushing the changes to GIT (so jazz will push them to bluemix)
What am I doing wrong?
From my understanding the server.xml from the starter is for your local Liberty runtime that you can also fire up from within the maven plugin. If you want to make changes to your bluemix Liberty feature set you can do so by setting cf environment variables.
See my recent blogpost on how I did this.
https://bluemixdev.wordpress.com/2016/02/07/bootstrap-a-websphere-liberty-webapp/
I added the following to the build script in my deployment pipeline.
cf set-env blueair-web JBP_CONFIG_LIBERTY “app_archive: {features: [servlet-3.1]}”
cf push “${CF_APP}”
Alternatively you can set the liberty feature set within your manifest, see this blogpost on how to do so: https://bluemixdev.wordpress.com/2016/02/21/specify-liberty-app-featureset-in-manifest/
If all you're trying to do is update the feature list, then setting the JBP_CONFIG_LIBERTY is the easiest way.
But if you're looking to provide more config in the server.xml then you'll need to provide a server package.
For example, for this case:
I can either:
I can issue a cf push myBluemixApp directly from the "videoServer" directory.
Or, package the server using the wlp/binserver package videoServer --include=usr command and then push the resulting zip file cf push myBluemixApp -p wlp/usr/servers/videoServer/videoServer.zip https://developer.ibm.com/bluemix/2015/01/06/modify-liberty-server-xml-configurations-ibm-bluemix/
Or, manually or using your build, create a wlp dir structure keeping only the files you want to upload as I've done in the deploy directory here: https://hub.jazz.net/project/rvennam/Microservices_Shipping/overview You can then push that directory as I'm doing (see manifest.yml). This will work with jazz/DevOps Services.
Packaging the server.xml within a war file is not the correct way.
I'm new to Capistrano and struggling a little to get started. A brief description of what I need to do:
git pull the latest code from our git repo, on a central build server. This build server's environment matches the deployment environment exactly. I need the code to be built here. I don't want to deploy a binary that was built on a Mac laptop, for example.
compile the binary on this machine.
deploy it from this machine to all the target machines.
There is a shared user we can all SSH into on the build machine to do the builds.
The build machine is behind a gateway machine, not directly accessible.
All of the deployment target machines also have this shared user and are also behind the gateway.
The deployed binary is a single executable, and there is an init script on the target machines. After deploying the binary and changing the symlink to it, restart the service via the init script.
Everyone has appropriate SSH keys and agent forwarding for all necessary tasks.
So in principle it seems rather simple, but Capistrano seems opinionated and a bit magical. As a result I'm not sure how to accomplish all of this. It seems like it wants to check out my code and copy it to the remote machines, for example without building it first.
I think I need to ignore all of Capistrano's default smarts and just make it run some shell commands on the appropriate servers. In pseudo-code:
ssh buildmachine via gateway "cd repo && git pull && make"
ssh targetmachine(s) via gateway "scp buildmachine:repo/binary .; <mv && symlink>; service foo restart"
Am I even using the right tool for the job? It seems a lot like a round peg in a square hole.
Can someone explain to me what the contents of the Capistrano configuration files should be, and what cap commands I'd run to accomplish this?
BTW, I've searched around and looked at questions like deploying with capistrano with remote git repo but without git running on production server and From manual pull on server to Capistrano
The question is rather old, but you never know when someone steps onto it in need of information...
First and formost, consider that Capistrano might just not be the right tool for the job you want to do.
That said, it is not impossible to accomplish what you expect. While in projects that deploy large amount of files and modify them (like css/js minify, js builds etc.) I would avoid it, in your case, you can consider runing a "deployment repository" and configure it in capistrano as the source. Your process would look like this :
run the local build with whatever tools you need
upload resulting binary to a deployment repository
run capistrano that will connect to application servers, fetch fresh binary from repository, perform any server side tasks required and symlink to "current"
As a side effect you end up with full history of deployed binaries
I have an MVC4 + EF4.0 .NET 4.5 project (say, MyProject) I'm able to run the project locally just fine. When I FTP deploy it to Azure Websites (not cloud service) it runs fine too. However, if I do a GIT deploy, the site 'runs' for the most part until it does some EF5.0 database operations. I get an exception Unable to load the specified metadata resource.
Upon debugging I noticed that if I:
GIT deploy the entire MVC4 project (as before)
FTP in and then replace bin\MyProject.dll with the bin\MyProject.dll file that I just built locally (Windows 8 x64, VS2012, Oct'12 Azure tools) after the GIT push (i.e. same source)
then the Azure hosted website runs just fine (even the EF5.0 database functionality portion).
The locally built .dll is about 5KB larger than the Azure GIT publish built one and both are 'Release' mode. It's obvious that the project as built after the GIT push (inside Azure) is being built differently than as on my own PC. I checked the portal and it's set to .NET 4.5. I'm also GIT pushing the entire solution folder (with just one project) and not just small bits and pieces.
When I load the locally built as well as the remotely built MyProject.dll files, I noticed the following difference(FrameworkDisplayName)
local: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5", FrameworkDisplayName = ".NET Framework 4.5"),
remote: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5", FrameworkDisplayName = ""),
Anyone knows why this is happening and what the fix might be?
Yes, this is a bug that will be fixed in the next release. The good news is that it's possible to work around it today:
First, you need to use a custom deployment script, per this post.
Then you need to change the MSBuild command line in the custom script per this issue.
Credit goes to David above for the pointers and hints. I voted him up but I'll also post the exact solution to the issue here. I've edited my original post because I found there was a major bug that I didn't notice until I started from scratch (moved GIT servers). So here is the entire process, worked for me.
Download Node.JS (it's needed even for .NET projects because the GIT deploy tools use it)
Install the azure-cli tool (open regular command prompt => npm install azure-cli -g)
In the command prompt, cd to the root of your repository (cd \projects\MyRepoRoot)
In there, type azure site deploymentscript --aspWAP PathToMyProject\MyProject.csproj -s PathToMySolution.sln (obviously adjust the paths as needed)
This will create the .deployment and deploy.cmd files
Now edit the deploy.cmd file, find the line starting with %MSBUILD_PATH% (will be just one)
Insert the /t:Build parameter. For example:
[Before] %MSBUILD_PATH% <blah blah> /verbosity:m /t:pipelinePreDeployCopyAllFilesToOneFolder
[After] %MSBUILD_PATH% <blah blah> /verbosity:m /t:Build /t:pipelinePreDeployCopyAllFilesToOneFolder)
Push to GIT (check the GIT output if everything went ok)
Browse to your website and confirm it works!
I'll be glad when it's fixed in the next revision so we won't maintain the build script