AzureDevops publish publishes too many zip files - azure-devops
Apologies if this is a stpuid question but I have been struggling with this for several weeks now and making no progress. I have a build step for my .Net project in Azure Devops with the following parameters:
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"
Although it says package as single file. I get the following in the drop folder:
Orchard.Autoroute.deploy-readme.txt
Orchard.Autoroute.deploy.cmd
Orchard.Autoroute.SetParameters.xml
Orchard.Autoroute.SourceManifest.xml
Orchard.Autoroute.zip
Orchard.Azure.deploy-readme.txt
Orchard.Azure.deploy.cmd
Orchard.Azure.MediaServices.deploy-readme.txt
Orchard.Azure.MediaServices.deploy.cmd
Orchard.Azure.MediaServices.SetParameters.xml
Orchard.Azure.MediaServices.SourceManifest.xml
Orchard.Azure.MediaServices.zip
Orchard.Azure.SetParameters.xml
Orchard.Azure.SourceManifest.xml
Orchard.Azure.zip
Orchard.Blogs.deploy-readme.txt
Orchard.Blogs.deploy.cmd
Orchard.Blogs.SetParameters.xml
Orchard.Blogs.SourceManifest.xml
Orchard.Blogs.zip
Orchard.Caching.deploy-readme.txt
Orchard.Caching.deploy.cmd
Orchard.Caching.SetParameters.xml
Orchard.Caching.SourceManifest.xml
Orchard.Caching.zip
Orchard.CodeGeneration.deploy-readme.txt
Orchard.CodeGeneration.deploy.cmd
Orchard.CodeGeneration.SetParameters.xml
Orchard.CodeGeneration.SourceManifest.xml
Orchard.CodeGeneration.zip
Orchard.Comments.deploy-readme.txt
Orchard.Comments.deploy.cmd
Orchard.Comments.SetParameters.xml
Orchard.Comments.SourceManifest.xml
Orchard.Comments.zip
Orchard.Conditions.deploy-readme.txt
Orchard.Conditions.deploy.cmd
Orchard.Conditions.SetParameters.xml
Orchard.Conditions.SourceManifest.xml
Orchard.Conditions.zip
Orchard.ContentPermissions.deploy-readme.txt
Orchard.ContentPermissions.deploy.cmd
Orchard.ContentPermissions.SetParameters.xml
Orchard.ContentPermissions.SourceManifest.xml
Orchard.ContentPermissions.zip
Orchard.ContentPicker.deploy-readme.txt
Orchard.ContentPicker.deploy.cmd
Orchard.ContentPicker.SetParameters.xml
Orchard.ContentPicker.SourceManifest.xml
Orchard.ContentPicker.zip
Orchard.ContentTypes.deploy-readme.txt
Orchard.ContentTypes.deploy.cmd
Orchard.ContentTypes.SetParameters.xml
Orchard.ContentTypes.SourceManifest.xml
Orchard.ContentTypes.zip
Orchard.Core.deploy-readme.txt
Orchard.Core.deploy.cmd
Orchard.Core.SetParameters.xml
Orchard.Core.SourceManifest.xml
Orchard.Core.zip
Orchard.CustomForms.deploy-readme.txt
Orchard.CustomForms.deploy.cmd
Orchard.CustomForms.SetParameters.xml
Orchard.CustomForms.SourceManifest.xml
Orchard.CustomForms.zip
Orchard.Dashboards.deploy-readme.txt
Orchard.Dashboards.deploy.cmd
Orchard.Dashboards.SetParameters.xml
Orchard.Dashboards.SourceManifest.xml
Orchard.Dashboards.zip
Orchard.DesignerTools.deploy-readme.txt
Orchard.DesignerTools.deploy.cmd
Orchard.DesignerTools.SetParameters.xml
Orchard.DesignerTools.SourceManifest.xml
Orchard.DesignerTools.zip
Orchard.DynamicForms.deploy-readme.txt
Orchard.DynamicForms.deploy.cmd
Orchard.DynamicForms.SetParameters.xml
Orchard.DynamicForms.SourceManifest.xml
Orchard.DynamicForms.zip
Orchard.Email.deploy-readme.txt
Orchard.Email.deploy.cmd
Orchard.Email.SetParameters.xml
Orchard.Email.SourceManifest.xml
Orchard.Email.zip
Orchard.Fields.deploy-readme.txt
Orchard.Fields.deploy.cmd
Orchard.Fields.SetParameters.xml
Orchard.Fields.SourceManifest.xml
Orchard.Fields.zip
Orchard.Forms.deploy-readme.txt
Orchard.Forms.deploy.cmd
Orchard.Forms.SetParameters.xml
Orchard.Forms.SourceManifest.xml
Orchard.Forms.zip
Orchard.ImageEditor.deploy-readme.txt
Orchard.ImageEditor.deploy.cmd
Orchard.ImageEditor.SetParameters.xml
Orchard.ImageEditor.SourceManifest.xml
Orchard.ImageEditor.zip
Orchard.ImportExport.deploy-readme.txt
Orchard.ImportExport.deploy.cmd
Orchard.ImportExport.SetParameters.xml
Orchard.ImportExport.SourceManifest.xml
Orchard.ImportExport.zip
Orchard.Indexing.deploy-readme.txt
Orchard.Indexing.deploy.cmd
Orchard.Indexing.SetParameters.xml
Orchard.Indexing.SourceManifest.xml
Orchard.Indexing.zip
Orchard.JobsQueue.deploy-readme.txt
Orchard.JobsQueue.deploy.cmd
Orchard.JobsQueue.SetParameters.xml
Orchard.JobsQueue.SourceManifest.xml
Orchard.JobsQueue.zip
Orchard.jQuery.deploy-readme.txt
Orchard.jQuery.deploy.cmd
Orchard.jQuery.SetParameters.xml
Orchard.jQuery.SourceManifest.xml
Orchard.jQuery.zip
Orchard.Layouts.deploy-readme.txt
Orchard.Layouts.deploy.cmd
Orchard.Layouts.SetParameters.xml
Orchard.Layouts.SourceManifest.xml
Orchard.Layouts.zip
Orchard.Lists.deploy-readme.txt
Orchard.Lists.deploy.cmd
Orchard.Lists.SetParameters.xml
Orchard.Lists.SourceManifest.xml
Orchard.Lists.zip
Orchard.Localization.deploy-readme.txt
Orchard.Localization.deploy.cmd
Orchard.Localization.SetParameters.xml
Orchard.Localization.SourceManifest.xml
Orchard.Localization.zip
Orchard.Media.deploy-readme.txt
Orchard.Media.deploy.cmd
Orchard.Media.SetParameters.xml
Orchard.Media.SourceManifest.xml
Orchard.Media.zip
Orchard.MediaLibrary.deploy-readme.txt
Orchard.MediaLibrary.deploy.cmd
Orchard.MediaLibrary.SetParameters.xml
Orchard.MediaLibrary.SourceManifest.xml
Orchard.MediaLibrary.zip
Orchard.MediaPicker.deploy-readme.txt
Orchard.MediaPicker.deploy.cmd
Orchard.MediaPicker.SetParameters.xml
Orchard.MediaPicker.SourceManifest.xml
Orchard.MediaPicker.zip
Orchard.MediaProcessing.deploy-readme.txt
Orchard.MediaProcessing.deploy.cmd
Orchard.MediaProcessing.SetParameters.xml
Orchard.MediaProcessing.SourceManifest.xml
Orchard.MediaProcessing.zip
Orchard.MessageBus.deploy-readme.txt
Orchard.MessageBus.deploy.cmd
Orchard.MessageBus.SetParameters.xml
Orchard.MessageBus.SourceManifest.xml
Orchard.MessageBus.zip
Orchard.Migrations.deploy-readme.txt
Orchard.Migrations.deploy.cmd
Orchard.Migrations.SetParameters.xml
Orchard.Migrations.SourceManifest.xml
Orchard.Migrations.zip
Orchard.Modules.deploy-readme.txt
Orchard.Modules.deploy.cmd
Orchard.Modules.SetParameters.xml
Orchard.Modules.SourceManifest.xml
Orchard.Modules.zip
Orchard.MultiTenancy.deploy-readme.txt
Orchard.MultiTenancy.deploy.cmd
Orchard.MultiTenancy.SetParameters.xml
Orchard.MultiTenancy.SourceManifest.xml
Orchard.MultiTenancy.zip
Orchard.OutputCache.deploy-readme.txt
Orchard.OutputCache.deploy.cmd
Orchard.OutputCache.SetParameters.xml
Orchard.OutputCache.SourceManifest.xml
Orchard.OutputCache.zip
Orchard.Packaging.deploy-readme.txt
Orchard.Packaging.deploy.cmd
Orchard.Packaging.SetParameters.xml
Orchard.Packaging.SourceManifest.xml
Orchard.Packaging.zip
Orchard.Pages.deploy-readme.txt
Orchard.Pages.deploy.cmd
Orchard.Pages.SetParameters.xml
Orchard.Pages.SourceManifest.xml
Orchard.Pages.zip
Orchard.Projections.deploy-readme.txt
Orchard.Projections.deploy.cmd
Orchard.Projections.SetParameters.xml
Orchard.Projections.SourceManifest.xml
Orchard.Projections.zip
Orchard.PublishLater.deploy-readme.txt
Orchard.PublishLater.deploy.cmd
Orchard.PublishLater.SetParameters.xml
Orchard.PublishLater.SourceManifest.xml
Orchard.PublishLater.zip
Orchard.Recipes.deploy-readme.txt
Orchard.Recipes.deploy.cmd
Orchard.Recipes.SetParameters.xml
Orchard.Recipes.SourceManifest.xml
Orchard.Recipes.zip
Orchard.Redis.deploy-readme.txt
Orchard.Redis.deploy.cmd
Orchard.Redis.SetParameters.xml
Orchard.Redis.SourceManifest.xml
Orchard.Redis.zip
Orchard.Resources.deploy-readme.txt
Orchard.Resources.deploy.cmd
Orchard.Resources.SetParameters.xml
Orchard.Resources.SourceManifest.xml
Orchard.Resources.zip
Orchard.Roles.deploy-readme.txt
Orchard.Roles.deploy.cmd
Orchard.Roles.SetParameters.xml
Orchard.Roles.SourceManifest.xml
Orchard.Roles.zip
Orchard.Rules.deploy-readme.txt
Orchard.Rules.deploy.cmd
Orchard.Rules.SetParameters.xml
Orchard.Rules.SourceManifest.xml
Orchard.Rules.zip
Orchard.Scripting.CSharp.deploy-readme.txt
Orchard.Scripting.CSharp.deploy.cmd
Orchard.Scripting.CSharp.SetParameters.xml
Orchard.Scripting.CSharp.SourceManifest.xml
Orchard.Scripting.CSharp.zip
Orchard.Scripting.deploy-readme.txt
Orchard.Scripting.deploy.cmd
Orchard.Scripting.Dlr.deploy-readme.txt
Orchard.Scripting.Dlr.deploy.cmd
Orchard.Scripting.Dlr.SetParameters.xml
Orchard.Scripting.Dlr.SourceManifest.xml
Orchard.Scripting.Dlr.zip
Orchard.Scripting.SetParameters.xml
Orchard.Scripting.SourceManifest.xml
Orchard.Scripting.zip
Orchard.Search.deploy-readme.txt
Orchard.Search.deploy.cmd
Orchard.Search.SetParameters.xml
Orchard.Search.SourceManifest.xml
Orchard.Search.zip
Orchard.SecureSocketsLayer.deploy-readme.txt
Orchard.SecureSocketsLayer.deploy.cmd
Orchard.SecureSocketsLayer.SetParameters.xml
Orchard.SecureSocketsLayer.SourceManifest.xml
Orchard.SecureSocketsLayer.zip
Orchard.Setup.deploy-readme.txt
Orchard.Setup.deploy.cmd
Orchard.Setup.SetParameters.xml
Orchard.Setup.SourceManifest.xml
Orchard.Setup.zip
Orchard.Tags.deploy-readme.txt
Orchard.Tags.deploy.cmd
Orchard.Tags.SetParameters.xml
Orchard.Tags.SourceManifest.xml
Orchard.Tags.zip
Orchard.TaskLease.deploy-readme.txt
Orchard.TaskLease.deploy.cmd
Orchard.TaskLease.SetParameters.xml
Orchard.TaskLease.SourceManifest.xml
Orchard.TaskLease.zip
Orchard.Taxonomies.deploy-readme.txt
Orchard.Taxonomies.deploy.cmd
Orchard.Taxonomies.SetParameters.xml
Orchard.Taxonomies.SourceManifest.xml
Orchard.Taxonomies.zip
Orchard.Templates.deploy-readme.txt
Orchard.Templates.deploy.cmd
Orchard.Templates.SetParameters.xml
Orchard.Templates.SourceManifest.xml
Orchard.Templates.zip
Orchard.Themes.deploy-readme.txt
Orchard.Themes.deploy.cmd
Orchard.Themes.SetParameters.xml
Orchard.Themes.SourceManifest.xml
Orchard.Themes.zip
Orchard.Tokens.deploy-readme.txt
Orchard.Tokens.deploy.cmd
Orchard.Tokens.SetParameters.xml
Orchard.Tokens.SourceManifest.xml
Orchard.Tokens.zip
Orchard.Users.deploy-readme.txt
Orchard.Users.deploy.cmd
Orchard.Users.SetParameters.xml
Orchard.Users.SourceManifest.xml
Orchard.Users.zip
Orchard.Warmup.deploy-readme.txt
Orchard.Warmup.deploy.cmd
Orchard.Warmup.SetParameters.xml
Orchard.Warmup.SourceManifest.xml
Orchard.Warmup.zip
Orchard.Web.deploy-readme.txt
Orchard.Web.deploy.cmd
Orchard.Web.SetParameters.xml
Orchard.Web.SourceManifest.xml
Orchard.Web.zip
etc and many more
In my release step I have an Azure App Service Deployment step with the following parameter:
$(System.DefaultWorkingDirectory)/**/Orchard.Web.zip
But it only deploys the code for the website, none of the modules of Orchard so the website does not work. I cannot specify this:
$(System.DefaultWorkingDirectory)/**/*.zip
Otherwise I get this error:
Error: More than one package matched with specified pattern: D:\a\r1\a***.zip. Please restrain the search pattern.
Currently the website publishes correctly if I publish from visual studio, but not through AzureDevops.
Do I need a deployment step for each module?
Why can't I do a deployment step just like in VS but through AzureDevops?
What combination of build and release steps do I need to do a WebDeploy which will deploy all of the code?
A tutorial would be ideal.
Edit:
I have also raised this issue here:
https://developercommunity.visualstudio.com/content/problem/408833/azuredevops-publish-publishes-too-many-zip-files.html
Related
remote server color doesn't look correct
I use ssh to login remote server quite often and on the server I'll use tools like emacs, git, tmux, etc. I noticed that the color on the remote server is very ugly compared with local machine. Especially for emacs, I want to use solarize theme but on the remote server it doesn't really show the correct color.
Does anyone know how to improve the color on remote server?
I am using item2 and when running infocmp I got the following results:
# Reconstructed via infocmp from file: /opt/sw/python/20201012/anaconda3/envs/i38/share/terminfo/x/xterm-256color
xterm-256color|xterm with 256 colors,
am, bce, ccc, km, mc5i, mir, msgr, npc, xenl,
colors#0x100, cols#80, it#8, lines#24, pairs#0x10000,
acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
clear=\E[H\E[2J, cnorm=\E[?12l\E[?25h, cr=\r,
csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
cud=\E[%p1%dB, cud1=\n, cuf=\E[%p1%dC, cuf1=\E[C,
cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
cvvis=\E[?12;25h, dch=\E[%p1%dP, dch1=\E[P, dim=\E[2m,
dl=\E[%p1%dM, dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K,
el1=\E[1K, flash=\E[?5h$<100/>\E[?5l, home=\E[H,
hpa=\E[%i%p1%dG, ht=^I, hts=\EH, ich=\E[%p1%d#,
il=\E[%p1%dL, il1=\E[L, ind=\n, indn=\E[%p1%dS,
initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
invis=\E[8m, is2=\E[!p\E[?3;4l\E[4l\E>, kDC=\E[3;2~,
kEND=\E[1;2F, kHOM=\E[1;2H, kIC=\E[2;2~, kLFT=\E[1;2D,
kNXT=\E[6;2~, kPRV=\E[5;2~, kRIT=\E[1;2C, ka1=\EOw,
ka3=\EOy, kb2=\EOu, kbs=^H, kc1=\EOq, kc3=\EOs, kcbt=\E[Z,
kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
kdch1=\E[3~, kend=\EOF, kent=\EOM, kf1=\EOP, kf10=\E[21~,
kf11=\E[23~, kf12=\E[24~, kf13=\E[1;2P, kf14=\E[1;2Q,
kf15=\E[1;2R, kf16=\E[1;2S, kf17=\E[15;2~, kf18=\E[17;2~,
kf19=\E[18;2~, kf2=\EOQ, kf20=\E[19;2~, kf21=\E[20;2~,
kf22=\E[21;2~, kf23=\E[23;2~, kf24=\E[24;2~,
kf25=\E[1;5P, kf26=\E[1;5Q, kf27=\E[1;5R, kf28=\E[1;5S,
kf29=\E[15;5~, kf3=\EOR, kf30=\E[17;5~, kf31=\E[18;5~,
kf32=\E[19;5~, kf33=\E[20;5~, kf34=\E[21;5~,
kf35=\E[23;5~, kf36=\E[24;5~, kf37=\E[1;6P, kf38=\E[1;6Q,
kf39=\E[1;6R, kf4=\EOS, kf40=\E[1;6S, kf41=\E[15;6~,
kf42=\E[17;6~, kf43=\E[18;6~, kf44=\E[19;6~,
kf45=\E[20;6~, kf46=\E[21;6~, kf47=\E[23;6~,
kf48=\E[24;6~, kf49=\E[1;3P, kf5=\E[15~, kf50=\E[1;3Q,
kf51=\E[1;3R, kf52=\E[1;3S, kf53=\E[15;3~, kf54=\E[17;3~,
kf55=\E[18;3~, kf56=\E[19;3~, kf57=\E[20;3~,
kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
kf61=\E[1;4P, kf62=\E[1;4Q, kf63=\E[1;4R, kf7=\E[18~,
kf8=\E[19~, kf9=\E[20~, khome=\EOH, kich1=\E[2~,
kind=\E[1;2B, kmous=\E[<, knp=\E[6~, kpp=\E[5~,
kri=\E[1;2A, mc0=\E[i, mc4=\E[4i, mc5=\E[5i, meml=\El,
memu=\Em, mgc=\E[?69l, oc=\E]104\007, op=\E[39;49m, rc=\E8,
rep=%p1%c\E[%p2%{1}%-%db, rev=\E[7m, ri=\EM,
rin=\E[%p1%dT, ritm=\E[23m, rmacs=\E(B, rmam=\E[?7l,
rmcup=\E[?1049l\E[23;0;0t, rmir=\E[4l, rmkx=\E[?1l\E>,
rmm=\E[?1034l, rmso=\E[27m, rmul=\E[24m,
rs1=\Ec\E]104\007, rs2=\E[!p\E[?3;4l\E[4l\E>, sc=\E7,
setab=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m,
setaf=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m,
sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p5%t;2%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
sgr0=\E(B\E[m, sitm=\E[3m, smacs=\E(0, smam=\E[?7h,
smcup=\E[?1049h\E[22;0;0t,
smglr=\E[?69h\E[%i%p1%d;%p2%ds, smir=\E[4h,
smkx=\E[?1h\E=, smm=\E[?1034h, smso=\E[7m, smul=\E[4m,
tbc=\E[3g, u6=\E[%i%d;%dR, u7=\E[6n,
u8=\E[?%[;0123456789]c, u9=\E[c, vpa=\E[%i%p1%dd,
Make sure your terminal type is set to xterm-256color in Preferences > Profiles > Terminal > Report Terminal Type. https://iterm2.com/faq.html Q: How do I get 256 color mode working in vim? A: The recommended way is to set your terminal type to xterm-256color in Preferences > Profiles > Terminal > Report Terminal Type. You may also need to put set background=light or set background=dark in your .vimrc.
How add Build Phases Run Script Phase in Ionic Appflow
I need to run a native Xcode build script during the iOS build in Ionic Appflow. For e.g I am using cordova-plugin-salesforce-snapins plugin and as per their documentation we need to run these scripts: select Build Phases create Run script paste this line of code $PODS_ROOT/ServiceSDK/Frameworks/ServiceCore.framework/prepare-framework It is possible in Xcode but there is no such way to achieve this in ionic Appflow. Please help me to configure this.
I use the npm plugin cordova-node-xcode, which allows you to write the build phase to the generated Xcode project file. Add "xcode": "^3.0.1" to your devDependencies in package.json.
Call a .js script AddBuildScript.js from a Cordova "after_build" hook in your config.xml:
<platform name="ios">
<hook src="AddBuildScript.js" type="after_build" />
Script AddBuildScript.js:
var xcode = require('xcode'),
fs = require('fs'),
projectPath = 'platforms/ios/MyProj.xcodeproj/project.pbxproj',
proj = xcode.project(projectPath);
proj.parse(function (err) {
var scriptName = 'My Script';
var buildPhases = proj.getPBXObject('PBXShellScriptBuildPhase');
if (JSON.stringify(buildPhases).match(scriptName)) {
console.log('Xcode project not updated - ' + scriptName + ' already exists')
} else {
var options = {shellPath: '/bin/sh', shellScript: '$PODS_ROOT/ServiceSDK/Frameworks/ServiceCore.framework/prepare-framework'};
proj.addBuildPhase([], 'PBXShellScriptBuildPhase', scriptName, proj.getFirstTarget().uuid, options);
fs.writeFileSync(projectPath, proj.writeSync());
console.log('Xcode project updated - added ' + scriptName);
}
});
java.lang.NoClassDefFoundError: Could not initialize class - spark/scala
I am new to spark/scala development. I am using maven to build my project and IDE is intelliJ. I am trying to query a hive table and then iterate over the resulting dataframe(using foreach). Here's my code:
try{
val DF_1 = hiveContext.sql("select distinct(address) from
test_table where trim(address)!=''")
println("number of rows: "+DF_1.count)
DF_1.foreach(x => {
val y =hiveContext.sql("select place from test_table where address='"+x(0).toString+"'")
if(y.count > 1){
println("Multiple place values for address: "+x(0).toString)
y.foreach(r => println(r))
println("*************")
}
})}
catch {case e: Exception => e.printStackTrace()}
With each iteration, I am Querying the same table to get another column, trying to see if there are multiple values of places for each address in test_table. I have no compilation errors and the application builds successfully. But, when I run the above code, I get the following error:
java.lang.NoClassDefFoundError: Could not initialize class xxxxxxxx
the application launches successfully, prints the count of rows in DF_1 and then fails with the above error at the foreach loop. I did a jar xvf on my jar and can see the main class - driver.class:
com/.../driver$$anonfun$1$$anonfun$apply$1.class
com/.../driver$$anonfun$1.class
com/.../driver$$anonfun$2.class
com/.../driver$$anonfun$3.class
com/.../driver$$anonfun$4.class
com/.../driver$$anonfun$5.class
com/.../driver$$anonfun$main$1$$anonfun$apply$1.class
com/.../driver$$anonfun$main$1$$anonfun$apply$2.class
com/.../driver$$anonfun$main$1$$anonfun$apply$3.class
com/.../driver$$anonfun$main$1.class
com/.../driver$$anonfun$main$10$$anonfun$apply$9.class
com/.../driver$$anonfun$main$10.class
com/.../driver$$anonfun$main$11.class
com/.../driver$$anonfun$main$12.class
com/.../driver$$anonfun$main$13.class
com/.../driver$$anonfun$main$14.class
com/.../driver$$anonfun$main$15.class
com/.../driver$$anonfun$main$16.class
com/.../driver$$anonfun$main$17.class
com/.../driver$$anonfun$main$18.class
com/.../driver$$anonfun$main$19.class
com/.../driver$$anonfun$main$2$$anonfun$apply$4.class
com/.../driver$$anonfun$main$2$$anonfun$apply$5.class
com/.../driver$$anonfun$main$2$$anonfun$apply$6.class
com/.../driver$$anonfun$main$2.class
com/.../driver$$anonfun$main$20.class
com/.../driver$$anonfun$main$21.class
com/.../driver$$anonfun$main$22.class
com/.../driver$$anonfun$main$23.class
com/.../driver$$anonfun$main$3$$anonfun$apply$7.class
com/.../driver$$anonfun$main$3$$anonfun$apply$8.class
com/.../driver$$anonfun$main$3.class
com/.../driver$$anonfun$main$4$$anonfun$apply$9.class
com/.../driver$$anonfun$main$4.class
com/.../driver$$anonfun$main$5.class
com/.../driver$$anonfun$main$6$$anonfun$apply$1.class
com/.../driver$$anonfun$main$6$$anonfun$apply$2.class
com/.../driver$$anonfun$main$6$$anonfun$apply$3.class
com/.../driver$$anonfun$main$6$$anonfun$apply$4.class
com/.../driver$$anonfun$main$6$$anonfun$apply$5.class
com/.../driver$$anonfun$main$6.class
com/.../driver$$anonfun$main$7$$anonfun$apply$1.class
com/.../driver$$anonfun$main$7$$anonfun$apply$2.class
com/.../driver$$anonfun$main$7$$anonfun$apply$3.class
com/.../driver$$anonfun$main$7$$anonfun$apply$4.class
com/.../driver$$anonfun$main$7$$anonfun$apply$5.class
com/.../driver$$anonfun$main$7$$anonfun$apply$6.class
com/.../driver$$anonfun$main$7$$anonfun$apply$7.class
com/.../driver$$anonfun$main$7$$anonfun$apply$8.class
com/.../driver$$anonfun$main$7.class
com/.../driver$$anonfun$main$8$$anonfun$apply$10.class
com/.../driver$$anonfun$main$8$$anonfun$apply$4.class
com/.../driver$$anonfun$main$8$$anonfun$apply$5.class
com/.../driver$$anonfun$main$8$$anonfun$apply$6.class
com/.../driver$$anonfun$main$8$$anonfun$apply$7.class
com/.../driver$$anonfun$main$8$$anonfun$apply$8.class
com/.../driver$$anonfun$main$8$$anonfun$apply$9.class
com/.../driver$$anonfun$main$8.class
com/.../driver$$anonfun$main$9$$anonfun$apply$11.class
com/.../driver$$anonfun$main$9$$anonfun$apply$7.class
com/.../driver$$anonfun$main$9$$anonfun$apply$8.class
com/.../driver$$anonfun$main$9$$anonfun$apply$9.class
com/.../driver$$anonfun$main$9.class
com/.../driver$.class
com/.../driver.class
I am not facing the error when I launch the job in local mode instead of yarn. What is causing the issue and how can it be corrected?
Any help would be appreciated, Thank you.
Looks like your your jar or some dependencies aren't distributed between worker nodes. In local mode it works because you have the jars in the place. In yarn mode you need to build a fat-jar with all dependencies include hive and spark libraries in it.
pmrep: [REP_55035] Not deploying DG - Dependent Session in source repository is required to deploy Workflow
In our current Project we are using Deployment Groups to migrate the code from DEV to TEST/PROD servers. We use Repository Manager to Create DG/Deploy the Code. As to reduce the effort on this, we want to perform this using PMREP command. Issue: I am deploying a DG which has 8 sessions which belong to a single Workflow ( lets call is wkf_ABC. This workflow contains 20 Sessions in TEST Env). Out of this 8 sessions , one session is new one and rest of the 8 sessions are ones which exist in ENV as well. I have added a)8 sessions with all dependencies and b) Workflow with no dependencies I am deploying it using two methods, one of them is using Repository Manger and other one is using PMREP command. 1) When using Repository Manger, I am able to deploy the DG to TEST env succesfully ( provided I compare the folders - checking the CHECK BOX in Select Compare Folders Option while Deploying the DG - or else its failing with the Error as shown in the screenshot) 2) When using the pmrep deploydeploymentgroup command with the same DG, the DG is not getting deployed showing the below error. Error: Dependent Session s_123 (id=xxxx) in source repository is required to deploy Workflow wkf_ABC, it is not available in the target repository and is not part of the deployment group. Copy failed, rolling back changes... Please help me if I am missing something More Inputs : 1) s_123 is not part of 8 sessions which are getting deployed and the session s_123 is already in Target repository ( ENV) 2) The Deployment config files which we have used with pmrep command and the one which I have used have same set of attributes. 3) I have seen there are some similar issues when using pmrep command like Object mismatch etc. other than that everything looks good. Systematic Deployment ( Repository Manager) <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE DEPLOYPARAMS SYSTEM "depcntl.dtd"> <DEPLOYPARAMS DEFAULTSERVERNAME = "" COPYPROGRAMINFO = "YES" COPYMAPVARPERVALS = "YES" COPYWFLOWVARPERVALS = "NO" COPYWFLOWSESSLOGS = "NO" COPYDEPENDENCY = "YES" LATESTVERSIONONLY = "NO" RETAINGENERATEDVAL = "YES" RETAINSERVERNETVALS = "YES" RETAINMAPVARPERVALS = "NO"> <DEPLOYGROUP> <APPLYLABEL SOURCELABELNAME = "" SOURCEMOVELABEL = "NO" TARGETLABELNAME = "" TARGETMOVELABEL = "NO"/> </DEPLOYGROUP> </DEPLOYPARAMS> Deployment Configurant file passed along with pmrep command <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE DEPLOYPARAMS SYSTEM "/dev1/usr/local/informatica/9.1.0/server/bin/depcntl.dtd"> <DEPLOYPARAMS DEFAULTSERVERNAME = "" COPYPROGRAMINFO = "YES" COPYMAPVARPERVALS = "YES" COPYWFLOWVARPERVALS = "NO" COPYWFLOWSESSLOGS = "NO" COPYDEPENDENCY = "YES" LATESTVERSIONONLY = "NO" RETAINGENERATEDVAL = "YES" RETAINSERVERNETVALS = "YES" RETAINMAPVARPERVALS = "NO"> <DEPLOYGROUP CLEARSRCDEPLOYGROUP="NO"> <OVERRIDEFOLDER SOURCEFOLDERNAME = "B_EDW_ODS_ICDS" SOURCEFOLDERTYPE = "LOCAL" TARGETFOLDERNAME = "B_EDW_ODS_ICDS" TARGETFOLDERTYPE = "LOCAL" MODIFIEDMANUALLY = "YES"/> . . .(extra lines for other folders) <APPLYLABEL SOURCELABELNAME = "" SOURCEMOVELABEL = "NO" TARGETLABELNAME = "" TARGETMOVELABEL = "NO"/> </DEPLOYGROUP> </DEPLOYPARAMS> Informatica(r) PMREP, version [9.1.0 HotFix6], build [496.0111], LINUX 64-bit
How to get list of TFS builds running at the moment from command line?
I'm trying to automate the deployment process, and as part of it, I need to run my release build from command line. I can do it, using command like .\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue It even returns some code for the queued build — Build queued. Queue position: 2, Queue ID: 11057. What I don't know, is how to get info about currently running builds, or about the state of my running build from powershell command line? The final aim is to start publishing after that build completes. I've already got all necessary powershell scripts to create the deployment package from the build results, zip it, copy to production and install there. All I need now — to know when my build succeedes.
This function will wait for a build with the Queue ID given by TFSBuild.exe:
function Wait-QueuedBuild {
param(
$QueueID
)
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Build.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Client')
$uri = [URI]"http://server-name:8080/tfs/project-collection"
$projectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($uri)
$buildServer = $projectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $buildServer.CreateBuildQueueSpec('*','*')
do {
$build = $buildServer.QueryQueuedBuilds($spec).QueuedBuilds| where {$_.Id -eq $QueueID}
sleep 1
} while ($build)
}
You can get the id returned by TFSBuild.exe, then call the function.
$tfsBuild = .\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
Wait-QueuedBuild [regex]::Match($tfsBuild[-1],'Queue ID: (?<id>\d+)').Groups['id'].Value
Using the work by E.Hofman available here it is possible to write a C# console app that uses TFS SDK and reveals if any build agent is currently running as follows:
using System;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
namespace ListAgentStatus
{
class Program
{
static void Main()
{
TfsTeamProjectCollection teamProjectCollection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://TFSServer:8080"));
var buildServer = teamProjectCollection.GetService<IBuildServer>();
foreach (IBuildController controller in buildServer.QueryBuildControllers(true))
{
foreach (IBuildAgent agent in controller.Agents)
{
Console.WriteLine(agent.Name+" is "+agent.IsReserved);
}
}
}
}
}
The parameter .IsReserved is what toggles to 'True' during execution of a build.
I 'm sorry my powershell skills are not good enough for providing with a PS variant of the above. Please take a look here, where the work by bwerks might help you do that.
# load classes for execution
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client") | Out-Null
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client") | Out-Null
# declare working variables
$Uri = New-Object System.Uri "http://example:8080/tfs"
# get reference to projection collection
$ProjectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($Uri)
# get reference to build server
$BuildServer = $ProjectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
# loop through the build servers
foreach($Controller in $BuildServer.QueryBuildControllers($true))
{
# loop through agents
foreach($BuildAgent in $Controller.Agents)
{
Write-Host "$($BuildAgent.Name) is $($BuildAgent.IsReserved)"
}
}