I'm trying to use application packages in the way they're described in this
https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
But I keep getting an error saying
application path not found.
Any ideas what could be wrong? Or
How do application packages work in the background, that might help me debug the error?
EDIT: I am trying to add an application package specific to my job manager task. I added the package as a zip file through Azure portal under the name JobManagerTask and version 1.0. Here is the code I'm using to reference it:
string taskID = "tasktest1";
// Obtain application package that has executables for job manager task
ApplicationPackageReference jobManagerApp = new ApplicationPackageReference { ApplicationId = "JobManagerTask", Version = "1.0" };
// Command Line
string commandLine = #"cmd /c %AZ_BATCH_APP_PACKAGE_JOBMANAGERTASK#1.0%\\JobManagerTask.exe";
// Create a CloudTask
CloudTask oneTask = new CloudTask(taskID, commandLine);
oneTask.ApplicationPackageReferences = new List<ApplicationPackageReference> { jobManagerApp };
// Provide elevated admin access to the task
oneTask.UserIdentity = new UserIdentity(new AutoUserSpecification(elevationLevel: ElevationLevel.Admin, scope: AutoUserScope.Task));
// Could add task resource files if needed here
await batchClient.JobOperations.AddTaskAsync(jobID, oneTask);
Coool, so i created a small barebone app. :) rest details are below and please feel free to ping me if I can helpout further.
so I tried with almost identical code like your's minus couple of flags like userIdentity and seems like I had my sample working fine, I think the error will only happen in case where the application package is not correctly refer'd. like if my *.exe reside in some diff dir structure etc. :)
I thought it will be a good idea to create a vanilla (i.e. from scratch application for you by taking one of the existing samples.) which might give you a chance to quickly take a look and see if you missed anything.
Please feel free to ping me and I will help you out to achieve your coal, I think its something very small like path is wrong etc. (which error message also suggest)
The Application reside here:
https://github.com/Tatsinnit/quick_sample_batchapppkgworking
Detail:
Detail is also there in the Readme for the git but as its a good practice in SO to detail everything here I will copy paste what I have written in readme here for you.
quick_sample_batchapppkgworking
Readme: barebone quick app:
Please note thta this app is nothing but a quick sample made based on the existing sample for DotNetTutorial.
Following code is generated just as a sample code for end to end app package working feature.
• https://learn.microsoft.com/en-us/azure/batch/
• https://learn.microsoft.com/en-us/azure/batch/batch-technical-overview
App Pacakges:
• https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
• https://azure.microsoft.com/en-us/blog/application-packages-and-task-dependencies-now-available-on-azure-batch/
The overview as how it works is fairly simple, when user uploads to adds an application package the package becomes available within node’s working directory (wd). The env var gets created to handle multiple updated versions of the app: (the timestamp is automatically part of the App pkg populated env var you dont need to do anything to handle this.)
set AZ_BATCH_APP_PACKAGE_TEST1#1.0=C:\user\tasks\applications\wd\test1\1.0\2017-07-14T21.45.45.765Z
Hence if user has correct package version all set and node has app pkg they can invoke that from whatever the need is for application package: (something like this)
string taskCommandLine = String.Format("cmd /c %AZ_BATCH_APP_PACKAGE_TEST1#1.0%\\ImageTest\\TaskApplication.exe");
The inside implementation is fairly neat as well.
Please note the reason:
%AZ_BATCH_APP_PACKAGE_TEST1#1.0%\\ImageTest\\TaskApplication.exe"
Is because my application package zip contains the TaskApplciaiton.exe under the following structure:
==> ==>
To add further: An application package is a .zip file that contains the application binaries and supporting files that are required for your tasks to run the application. Each application package represents a specific version of the application.+
You can specify application packages at the pool and task levels. You can specify one or more of these packages and (optionally) a version when you create a pool or task.+
• Pool application packages are deployed to every node in the pool. Applications are deployed when a node joins a pool, and when it is rebooted or reimaged.
Pool application packages are appropriate when all nodes in a pool execute a job's tasks. You can specify one or more application packages when you create a pool, and you can add or update an existing pool's packages. If you update an existing pool's application packages, you must restart its nodes to install the new package.
• Task application packages are deployed only to a compute node scheduled to run a task, just before running the task's command line. If the specified application package and version is already on the node, it is not redeployed and the existing package is used.
Task application packages are useful in shared-pool environments, where different jobs are run on one pool, and the pool is not deleted when a job is completed. If your job has fewer tasks than nodes in the pool, task application packages can minimize data transfer since your application is deployed only to the nodes that run tasks.
The sample attached contains both pool level level as well the task level demo.
Steps:
At first add a new Application Package into my batch account: you can do that via portal. (the git project has test1.zip along with this git sample console app.
Then open your DotNetTurorial solution:
Fill in these info for the batch account credentials or any storage account in use for your credentials correctly:
Hit start the barebone.cs is set as the start project, ** please note you might need to change your *.proj file, because in my local all nugets were getting sourced from c:\cxcache
Please also note, there will be prompt to delete the job and pool, if you want to checkout the return result of this app, please keep the job and pool and then go inside node inside that pool and checkout stdout.txt file for the txt printed. (Note: you probably want to delete job and pool from the portal once you are done.)
The screenshots from my successful run are below:
So I was able to see the Test Success getting printed in my stdout.txt inside node from the TaskApplication.exe which was part of this application package.
The code used in this sample barebone app is reused fomr the sample existing here:
https://github.com/Azure/azure-batch-samples/tree/master/CSharp/ArticleProjects/DotNetTutorial.
Other friendly screenshots:
Related
I have a basic Azure Function app. When I try to publish the app, I receive an error that says "error : The attempt to publish the ZIP file through https://... failed with HTTP status code RequestTimeout.".
This app is a .NET Standard app. I followed the instructions here. The difference is, my app has an Event Hub Trigger instead of the Http Trigger shown in the documentation. I don't understand why i'm getting a Timeout during deployment. I also don't know how to get past this.
What am I doing wrong?
Update
Here are the logs.
1>------ Build started: Project: MyProject.Functions, Configuration: Release Any CPU ------
1>MyProject.Functions -> C:\MyProject\MyProject.Functions\bin\Release\netcoreapp2.1\bin\MyProject.Functions.dll
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
Publish Started
MyProject.Functions -> C:\MyProject\MyProject.Functions\bin\Release\netcoreapp2.1\bin\MyProject.Functions.dll
MyProject.Functions -> C:\MyProject\MyProject.Functions\obj\Release\netcoreapp2.1\PubTmp\Out\
Publishing C:\MyProject\MyProject.Functions\obj\Release\netcoreapp2.1\PubTmp\MyProject.Functions - 20181101105531356.zip to https://my-project.scm.azurewebsites.net/api/zipdeploy...
C:\Users\me\.nuget\packages\microsoft.net.sdk.functions\1.0.23\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Publish.ZipDeploy.targets(42,5): error : The attempt to publish the ZIP file through https://my-project.scm.azurewebsites.net/api/zipdeploy failed with HTTP status code RequestTimeout. [C:\MyProject\MyProject.Functions\MyProject.Functions.csproj]
According to this:
https://github.com/projectkudu/kudu/wiki/Deploying-from-a-zip-file
you should be able to pass ?isAsync=true to the zipdeploy url (so it would be: 'https://my-project.scm.azurewebsites.net/api/zipdeploy?isAsync=true'
This requests resolves faster without a timeout and then you can grab the location header from the response, which you can poll to see the status of your deployment.
In my case this error was because of the version of packages in my .csproj file. After updating them there was not error and the publish was successful.
I faced this recently and spent 2 complete days trying to fix it. Tried most of the solutions suggested here and on other posts.
What finally worked for me is removing my Publish settings and creating a new one by uploading a brand new .PublishSettings file.
How to get .PublishSettings file?
On Azure Portal, on your Function App, click on "Get Publish Profile"
And will automatically start downloading it.
How to Upload Publish Profile?
When trying to Publish the project from Visual Studio, click on New -> Select "Import Profile"
And Browse your .PublishSettings file.
Then, just select this new profile (if it's not selected already), and click on Publish button as you would usually do.
In my case, it was an issue with two things:
1] Visual Studio and Azure are flaky. Timeouts in a working scenario are still somewhat regular, on a bad day happening about 50-75% of the time for me. This is with an 80mb function app, not super big and I have gigabit Internet.
2] Someone deleted the file share for the storage. I had to fix WEBSITE_CONTENTAZUREFILECONNECTIONSTRING to point to the right storage connection string, and I had to update WEBSITE_CONTENTSHARE to point to a valid file share name, which I had to create in the storage resource group matching WEBSITE_CONTENTAZUREFILECONNECTIONSTRING connection string.
If you are using a development and production function slot, I would suggest to make WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE deployment slot settings, that way you can link to a production and development storage environment. This is especially handy if you are using tables or blob storage and don't want to have to prefix or suffix all your table names or keys. In my opinion these two settings should be slots by default.
Once I did these changes I could publish, still dealing with the intermittent timeouts.
The error messaging with Azure function publishing is bad to non-existant, with any kind of configuration or resource errors simply causing a timeout error.
I got the same issue when using Visual Studio. Very frustrating.
But then I just used the zip file that VS created and used
az functionapp deployment source config-zip -g <resource_group> -n \
<app_name> --src <zip_file_path>
to publish.
You can find more options in
https://learn.microsoft.com/en-us/azure/azure-functions/deployment-zip-push
I got the same issue recently.
I'm not sure if they are related, but it started working fine after updating the NuGet package "Microsoft.NET.Sdk.Functions" to v3.0.7.
Changing the profile to use WebDeploy was the only way i could update my Azure Function.
When downloading the Profiles from the Azure Portal, and importing to VS - i noticed it imported 2 profiles. 1 for Zip, and another for Web Deploy method for uploading.
Trying the Zip publish profile, failed, but the WebDeploy 2nd Profile - did work and update perfectly.
Our code building process is done via an http server which starts the build process after receiving a project uuid from the build command. Once the server starts the compilation, GCC compatible output can be fetched from it.
Note: only my extension is aware of the project uuid which is different per workspace.
AFAIU I can implement it by:
programmatically adding a task which will call a script with the correct workspace uuid. Is this possible?
Having my extension manage the build process. This seems to be far from supported.
Bottom line, I'm trying to avoid asking the user to add anything to the configuration files and I want to completely manage the build process.
Thanks!
As I didn't find a suitable only vscode solution I did the following:
Defined a helper script which I executed as the task. The helper script was respojnsible for the communication against the HTTP server.
I registered the task using vscode.workspace.registerTaskProvider API, and made sure to register it only after figuring out the UUID.
Then in the task itself I executed the helper script.
(A nice task register example can be found here: https://github.com/Microsoft/vscode-extension-samples/tree/master/task-provider-sample)
Recently, I was assigned the task to create a deployment package for an application which btw, I'm totally new at. So far, so good.. Now there is a requirement to extract files from a zip file which will be bundled with the setup file. So, I had to write custom actions in the 'Commit' section of the Installer class. I added the Installer class in a new project of type 'Class Library' under the same solution. I wrote the code after 'base.Commit(savedState)'.
I tried showing MessageBox at the event entry point, used Debugger.Launch(), Debugger.Break() but somehow, no matter what I do, it seems that the custom action is not willing to be hit at all and the application just installs itself. I searched a lot of sites and blogs but no help so far.
I've assigned my installer class (SampleApp.exe, in my case) to all the Custom Action's modes (Install, Commit, Rollback and Uninstall) in the Deployment project. Any help.
P.S. I'm using a Visual Studio 2010 setup project.
Thanks, in advance!
You should probably be trying a class library Dll, not an executable (which is typically for something like a service).
You don't need it all the nodes if all you're doing is calling at Commit. And why Commit? Install is just the same in most cases.
If you're not seeing a MessageBox then probably your CA isn't being called, and that may because it's not a class library. Note that your CA is not running in the interactive user context - it's being called from an msiexec process running with the system account, so you must be very explicit about (say) the path to the zip file, and any user profile folders will probably fail because the system account doesn't really have them.
What files are these and where are they going on disk? If they are user profile files you can install the zip files to a per machine location and then have the application itself unzip the files to the desired location on first launch. Unzipping from within your setup is not good practice - it is error prone and bad design.
Using the application allows proper exception handling and interactivity (the user can be informed if something goes wrong). Set some registry flags in HKCU when you have completed the unzipping so it doesn't happen more than once, and perform the unzip once per user.
I have a puppet master sources my application builds into a master folder. for eg. xxxxx_v1.0.0.zip and yyyyy_v1.0.8.zip [xxxxx gets deployed to a ser of servers and yyyyy to another set of servers].
What is the best way to handle sourcing on puppet master on new versions of my application builds, without editing the .pp files on the master to reference the new build number on the filename, preferably, automatic.
Thanks
A good way to build a suitable package for your operating system instead. Puppet can use those with
package { 'application-x': ensure => latest }
Failing that, you solve this
on the agent side, by fetching your application metadata from somewhere, e.g. with an exec of wget, then having it run a script to perform the deployment if necessary
on the master side using an ENC like the Puppet Dashboard, or better yet, Hiera, to hold your latest version information
If you really want to do this through Puppet's fileserver without touching any metadata and just dropping the files in your modules, you can try with the generate function.
$latest_zip_application_x = generate("/usr/local/bin/find_latest application_x")
file { 'application_x.zip':
...
source => "puppet:///modules/application_x/path/to/$latest_zip_application_x",
}
where /usr/local/bin/find_latest is a script that will find the most recent version of your package and write it to stdout.
This is pretty horrible practice though - you are really not catering to Puppet's strengths with constructs like these.
I am using proget to upload packages, I am manually uploading from disk, but when I go to check if the package exists in the feed it isn't there. When I logon to the server which is hosting proget and go to the PackagesRootPath I can see the package is indeed on the server!
Any ideas why it's not showing up in the feed?
p.s. I have restarted the website/application pool and ProGet service and still doesn't work.
If you're not seeing any packages in the web application (and you've verified that they are, in fact, in the right place on disk), this means that the packages aren't getting indexed by the ProGet Service.
Since you've already restarted the ProGet web service, it's likely a problem with the individual package.
Check to see if there are "indexing errors" in the admin section; this will give some insight into what the problem might be. Often times, the file name does not match the package name/version; this is a requirement. If you're package is named MyFoo and is version 3.0.1, it must be MyFoo.3.0.1.nupkg and have an appropriately named MyFoo.nuspec within it.
If there are no errors logged, then you can try to run the service interactively. Simply stop the Windows service, then run the .exe file and select the appropriate option to run.
Another option to verify that the indexing is working OK is to pull a package from a remote connector (like JQuery or something), then drop that package in another feed (that doesn't use a connector).