Azure Batch default application package version - azure-batch

According to the Microsoft Documentation, in Azure Batch without the version specified the default version of an application will be deployed.
In my Azure Batch Account I have uploaded an application "MyApp" and set the default version, lets say version "1.0".
When I create a new Pool (in .NET) if I set the ApplicationPackageReferences omitting the version, i.e:
myCloudPool.ApplicationPackageReferences = new List<ApplicationPackageReference>
{
new ApplicationPackageReference
{
ApplicationId = "MyApp"
}
};
The nodes will get the status "Unusable".
If I do the same, but at task level, then the default application is deployed successfully to the node.
Why is that?

Thank you Olandese, I came across this specific case only in the case for default version at pool level and the fix is under its way, I will keep you posted when it will get released.
In the mean time there are few options or at pool level you can do so by using the version parameter.
Task level packages with default, which you are already mentioned above.
Use of the version parameter to pass the default version.
`
Sample:
new List< ApplicationPackageReference >
{
new ApplicationPackageReference(appID, version: appVersion),
},
`
Thanks for your patience, apologies for the inconvenience, will update you once the release is out.
Further addition: If you have faster changing default version
I would do it this way: (With appId as constant and the version as the dynamic version)
By doing this your code pretty much guarantees that node will go and grab the specified version mentioned if its there. Hope this helps.
`
new List<ApplicationPackageReference>
{
new ApplicationPackageReference(appID, version: appVersion),
},

Related

Azure batch Application package not getting copied to Working Directory of Task

I have created Azure Batch pool with Linux Machine and specified Application Package for the Pool.
My command line is
command='python $AZ_BATCH_APP_PACKAGE_scriptv1_1/tasks/XXX/get_XXXXX_data.py',
python3: can't open file '$AZ_BATCH_APP_PACKAGE_scriptv1_1/tasks/XXX/get_XXXXX_data.py':
[Errno 2] No such file or directory
when i connect to node and look at working directory non of the Application Package files are present there.
How do i make sure that files from Application Package are available in working directory or I can invoke/execute files under Application Package from command line ?
Make sure that your async operation have proper await in place before you start using the package in your code.
Also please share your design \ pseudo-code scenario and how you are approaching it as a design?
Further to add:
Seems like this one is pool level package.
The error seems like that the application env variable is either incorrectly used or there is some other user level issue. Please checkout linmk below and specially the section where use of env variable is mentioned.
This seems like user level issue because In case of downloading the package resource, if there will be an error it will be visible to you via exception handler or at the tool level is you are using batch explorer \ Batch-labs or code level exception handling.
https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
Reason \ Rationale:
If the pool level or the task application has error, an error-list will come back if there was an error in the application package then it will be returned as the UserError or and AppPackageError which will be visible in the exception handle of the code.
Key you can always RDP into your node and checkout the package availability: information here: https://learn.microsoft.com/en-us/azure/batch/batch-api-basics#connecting-to-compute-nodes
I once created a small sample to help peeps around so this resource might help you to checkeout the use here.
Hope rest helps.
On Linux, the application package with version string is formatted as:
AZ_BATCH_APP_PACKAGE_{0}_{1}
On Windows it is formatted as:
AZ_BATCH_APP_PACKAGE_APPLICATIONID#version
Where 0 is the application name and 1 is the version.
$AZ_BATCH_APP_PACKAGE_scriptv1_1 will take you to the root folder where the application was unzipped.
Does this "exact" path exist in that location?
tasks/XXX/get_XXXXX_data.py
You can see more information here:
https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
Edit: Just saw this question: "or can I invoke/execute files under Application Package from command line"
Yes you can invoke and execute files from the application package directory with the environment variable above.
If you type env on the node you will see the environment variables that have been set.

Azure Service Fabric - change config settings for a deployed Application

How do I change settings for a deployed application in Service Fabric?
I have a provisioned cluster and an application deployed to the cluster with two applications. I would like to be able to change my services' settings and have them pick up those changes, but I don't see how I can do that.
Previously, we've done all of our services with worker roles in Cloud Services, and the portal allows for changing configurations, but it does not appear to do so for Service Fabric. From the Service Fabric Explorer I can drill down to the service, go to MANIFEST and view the XML with the settings. I just don't see a way to edit or change it. I've struggled finding anything in the SF documentation addressing this.
The portal doesn't expose a way to do this. It needs to be done via an upgrade of the application. Just change the settings in your settings XML file and perform an upgrade. In the VS publish dialog for your application project, you can update your version numbers appropriately by changing the config package version which will automatically bubble up to update the containing service and application versions.
Building on Matt Thalman's answer, here's documentation on modifying the settings in the application or service manifest XML files, updating the version numbers, and performing an application upgrade: Service Fabric application upgrade tutorial using Visual Studio. You can also perform the app upgrade using PowerShell.
Additional to above answers, adding some powershell code..
we may use below powershell code to connect to Service Fabric from powershell and get the application parameters and then update specific parameter and re deploy..
### Change the connection here (from Profile-Cloud.xml
$ConnectArgs = #{
ConnectionEndpoint="devxxxxxx.westus.cloudapp.azure.com:19000"
X509Credential="true"
ServerCertThumbprint="52BFxxxxxxxxxx"
FindType="FindByThumbprint"
FindValue="EF3A2xxxxxxxxxxxxxx"
StoreLocation="CurrentUser"
StoreName="My"
}
Connect-ServiceFabricCluster #ConnectArgs
$myApplication = Get-ServiceFabricApplication -ApplicationName fabric:/ABC.MyService
$appParamCollection = $myApplication.ApplicationParameters
### Update your parameter here..
$applicationParameterMap.ElasticSearch_Username="sachin2"
$applicationParameterMap = #{}
foreach ($pair in $appParamCollection)
{
$applicationParameterMap.Add($pair.Name, $pair.Value);
}
### Start Udpating
Start-ServiceFabricApplicationUpgrade -ApplicationName $myApplication.ApplicationName.OriginalString -ApplicationTypeVersion $myApplication.ApplicationTypeVersion -ApplicationParameter $applicationParameterMap -Monitored -FailureAction Rollback -ForceRestart $true
### Check the status until it is Ready
(Get-ServiceFabricApplication -ApplicationName fabric:/ABC.MyService).ApplicationStatus
### Check the parameters to confirm those're updated
Get-ServiceFabricApplication -ApplicationName fabric:/ABC.MyService
You may change or remove the -ForceRestart as per your requriements

Configuring spring-xd to use oracle as job repository

I want to run spring xd with Oracle(11g) which i already have in my environment. Currently my first concern is the jobs UI (my database has existing data of job executions that were performed by spring-batch and i simply want to display the details of those executions).
i'm using spring-xd-1.0.0.M5. I followed the instructions in the reference guide and i changed application.yml to have the following:
spring:
datasource:
url: jdbc:oracle:oci:MY_USERNAME/MYPWD#//orarmydomain.com:1521/myservice
username: MY_USERNAME
password: MYPWD
driverClassName: oracle.jdbc.OracleDriver
profiles:
active: default,oracle
i modified also batch-jdbc.properties to have the database configuration similar to the above.
Yet, when i start xd-singlnode.bat (or either xd-admin.bat) it seems like it ignores my oracle configuration and still uses the default hsqldb.
what am i doing wrong?
Thanks
The likely reason is that we did not upgrade the windows .bat scripts to take advantage of the property overriding via xd-config.yml. If you go into the unix script for xd-singlenode you will see that when java is invoked there there is an option
-Dspring.config.location=$XD_CONFIG
you can for now hardcode your location of that file, use file: as the prefix.
Also, The UI right now is very primitive, you will not be able to see many details about the job execution. There are however many job related commands you can execute in the shell and there is only one gap regarding step execution information as compared to what is available via spring-batch-admin.
The issue to watch for this is https://jira.springsource.org/browse/XD-1209 and it is schedule for the next milestone release.
Let me know how it goes, thanks!
Cheers,
Mark

change config/newrelic.yml path

I'm using the new relic ruby agent with symfony 2.1 and capifony.
I'd like to be able to tell the new relic agent that the config file lives under app/config/newrelic.yml instead of config/newrelic.yml
Is this possible?
You have to set NRCONFIG environment variable and you can do it directly in cap call this way:
cap newrelic:notice_deployment NRCONFIG=app/config/newrelic.yml
I would like to set it in deployment configuration and according to cap documentation this should work:
set :default_environment, {
"NRCONFIG" => "app/config/newrelic.yml"
}
but it doesn't. ;o(

Upgrading from DeploymentService to ProfileService in JBoss 5.1.0 GA

In Jboss 5.1 the Profile Service does what the Deployment Service was doing in Jboss 4.x.In Jboss 4.x I was using the Deployment Service to create a datasource "on-the-fly" and I was wondering if I could do the same thing using the Profile Service (since Deployment Service doesn't exist any more in Jboss 5.x).
Does anyone know a practical guid on using ProfileService?
Thank you ,
Regards.
I don't know of any guide but I can provide you with my experience using the Profile Service and a few links to JBoss wiki pages on this topic. I'd like to post more links but the spam protection doesn't allow me to post more than two, but you should easily find the other pages in the wiki on the ProfileService. Don't be suprised in case you don't find much, there isn't more.
ProfileService ManagementView
http://community.jboss.org/wiki/ProfileServiceManagementView
ProfileService DeploymentTemplates
http://community.jboss.org/wiki/ProfileServiceDeploymentTemplates
There you'll find usefull information about the ProfileService but no detailed information is available in the jboss wiki as far as I can tell.
In order to create Datasources on the fly you can use the DeploymentTemplates (also for creating message queues and topics) The last link provides you with information on how to use the templates but not with all the template names and their properties. You can list them programatically though.
// Get all Templates
for(String template : mgtView.getTemplateNames())
{
System.out.println("=========================================");
System.out.println("Listing properties for template: "+template);
DeploymentTemplateInfo info = mgtView.getTemplate(template);
for(String prop : info.getProperties().keySet())
System.out.println("- "+prop);
}
In order to get the ManagementView (mgtView) from an external java programm you can use something similiar to this:
// set security policy
System.setProperty("java.security.policy", "<path_to_policy_file>");
System.setSecurityManager( new RMISecurityManager() ) ;
// set initial context properties
Hashtable<String, String> env = new Hashtable<String, String>();
env.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
env.put("java.naming.provider.url","jnp://localhost:1099");
env.put("java.naming.factory.url.pkgs","org.jboss.naming:org.jnp.interfaces");
ctx = new InitialContext(env);
// login to JBoss
SecurityClient client = SecurityClientFactory.getSecurityClient();
client.setSimple("admin", "admin");
client.login();
// get ProfileService and ViewManager
ProfileService ps = (ProfileService) ctx.lookup("ProfileService");
mgtView = ps.getViewManager();
What you want to get is the Java Naming Conext (InitialContext). In order to do that you'll need a security policy (you can use the java.policy file which is located in JBOSS_HOME/server/SERVER_DIR/conf/),security manager and environment properties to get the context. The java.naming.provider.url specifies the location of the JBoss naming service (default port is 1099).
Usually you would have to authenticate at this point which is done with the SecurityClient.
Finally you can use the context to grap the ProfileService.
At this point most of the anoying stuff is done und you can start playing around.
getViewManager() returns the ViewManager with which you can create datasources on the fly and getDeploymentManager() will give you the DeploymentManager with which you can deploy, undeploy, start, stop applications and other deployments.
The libraries you'll need to do that are located in
JBOSS_HOME/client
JBOSS_HOME/lib
JBOSS_HOME/common/lib
I've read several times that including the jbossall-client.jar in the client directory should be enough but that's actually not true. You need libraries from all three directories as far as I can tell (couldn't do it without referencing all of them at least). I haven't figured out which exact jars you need though...
IMPORTANT: The ProfileService in Jboss 5 Community Edition has some bugs though which got fixed in JBoss 6. I'd either suggest using a newer JBoss version or the Enterprise Edition.