In our current Project we are using Deployment Groups to migrate the code from DEV to TEST/PROD servers. We use Repository Manager to Create DG/Deploy the Code. As to reduce the effort on this, we want to perform this using PMREP command.
Issue:
I am deploying a DG which has 8 sessions which belong to a single Workflow ( lets call is wkf_ABC. This workflow contains 20 Sessions in TEST Env). Out of this 8 sessions , one session is new one and rest of the 8 sessions are ones which exist in ENV as well. I have added
a)8 sessions with all dependencies and
b) Workflow with no dependencies
I am deploying it using two methods, one of them is using Repository Manger and other one is using PMREP command.
1) When using Repository Manger, I am able to deploy the DG to TEST env succesfully ( provided I compare the folders - checking the CHECK BOX in Select Compare Folders Option while Deploying the DG - or else its failing with the Error as shown in the screenshot)
2) When using the pmrep deploydeploymentgroup command with the same DG, the DG is not getting deployed showing the below error.
Error: Dependent Session s_123 (id=xxxx) in source repository is required to deploy Workflow wkf_ABC, it is not available in the target repository and is not part of the deployment group.
Copy failed, rolling back changes...
Please help me if I am missing something
More Inputs :
1) s_123 is not part of 8 sessions which are getting deployed and the session s_123 is already in Target repository ( ENV)
2) The Deployment config files which we have used with pmrep command and the one which I have used have same set of attributes.
3) I have seen there are some similar issues when using pmrep command like Object mismatch etc. other than that everything looks good.
Systematic Deployment ( Repository Manager)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE DEPLOYPARAMS SYSTEM "depcntl.dtd">
<DEPLOYPARAMS DEFAULTSERVERNAME = "" COPYPROGRAMINFO = "YES" COPYMAPVARPERVALS = "YES" COPYWFLOWVARPERVALS = "NO" COPYWFLOWSESSLOGS = "NO" COPYDEPENDENCY = "YES" LATESTVERSIONONLY = "NO" RETAINGENERATEDVAL = "YES" RETAINSERVERNETVALS = "YES" RETAINMAPVARPERVALS = "NO">
<DEPLOYGROUP>
<APPLYLABEL SOURCELABELNAME = "" SOURCEMOVELABEL = "NO" TARGETLABELNAME = "" TARGETMOVELABEL = "NO"/>
</DEPLOYGROUP>
</DEPLOYPARAMS>
Deployment Configurant file passed along with pmrep command
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE DEPLOYPARAMS SYSTEM "/dev1/usr/local/informatica/9.1.0/server/bin/depcntl.dtd">
<DEPLOYPARAMS DEFAULTSERVERNAME = "" COPYPROGRAMINFO = "YES" COPYMAPVARPERVALS = "YES" COPYWFLOWVARPERVALS = "NO" COPYWFLOWSESSLOGS = "NO" COPYDEPENDENCY = "YES" LATESTVERSIONONLY = "NO" RETAINGENERATEDVAL = "YES" RETAINSERVERNETVALS = "YES" RETAINMAPVARPERVALS = "NO">
<DEPLOYGROUP CLEARSRCDEPLOYGROUP="NO">
<OVERRIDEFOLDER SOURCEFOLDERNAME = "B_EDW_ODS_ICDS" SOURCEFOLDERTYPE = "LOCAL" TARGETFOLDERNAME = "B_EDW_ODS_ICDS" TARGETFOLDERTYPE = "LOCAL" MODIFIEDMANUALLY = "YES"/>
.
.
.(extra lines for other folders)
<APPLYLABEL SOURCELABELNAME = "" SOURCEMOVELABEL = "NO" TARGETLABELNAME = "" TARGETMOVELABEL = "NO"/>
</DEPLOYGROUP>
</DEPLOYPARAMS>
Informatica(r) PMREP, version [9.1.0 HotFix6], build [496.0111], LINUX 64-bit
Related
Starting with this buildbot master config:
# -*- python -*-
# ex: set filetype=python:
from buildbot.plugins import *
c = BuildmasterConfig = {}
c['workers'] = [worker.Worker("example-worker", "pass")]
c['protocols'] = {'pb': {'port': 9989}}
c['change_source'] = []
c['schedulers'] = []
c['schedulers'].append(schedulers.ForceScheduler(
name="force",
builderNames=["awesomesauce", "runtests"]))
factory = util.BuildFactory()
factory.addStep(steps.ShellCommand(command=['echo', 'hi']))
c['builders'] = []
c['builders'].append(
util.BuilderConfig(name="runtests",
workernames=["example-worker"],
factory=factory))
c['builders'].append(
util.BuilderConfig(name="awesomesauce",
workernames=["example-worker"],
factory=factory))
c['status'] = []
c['title'] = "Cool Beans"
c['titleURL'] = "http://example.com"
c['buildbotURL'] = "http://localhost:8010/"
c['www'] = dict(port=8010,
plugins=dict(waterfall_view={}, console_view={}))
c['db'] = {
'db_url' : "sqlite:///state.sqlite",
}
If I change the ForceScheduler to have the following:
builderNames=['awesomesauce', 'runtests', 'coolbeans']
And then simply copy the builders append block and change the name:
c['builders'].append(
util.BuilderConfig(name="coolbeans",
workernames=["example-worker"],
factory=factory))
Then when I do:
buildbot reconfigure master
It says that it added new builders, however when I go to the builders page it only lists the worker for for my original two builders, and the force button is missing from the builder page. Ruh roh.
Is there a way to fix the worker/builder link without completely restarting my buildbot master?
For the Force button, I'd say you have to also reconfigure the ForceScheduler, which is not part of your description.
For the worker assignment it looks like indeed this is a bug of our reconfiguration algorithm
http://trac.buildbot.net/ticket/3629#ticket
This bug only affects the UI. The builder is still configured with this worker, and should be able to start builds with that worker
We currently are running the latest version of GitLab (v8.0.1) which is installed using the Omnibus package and trying to enable the new "reply-to" feature but nothing is happening.
We followed these instructions:
http://doc.gitlab.com/ce/incoming_email/README.html (specifically the Gmail instructions). We configured a new Gmail account with lesser-security and we also use the SMTP configuration.
The email, when replied to, is being sent to the GMail account but from there nothing is happening. The doco seems a little sparse but is GitLab supposed to pick that email up (via IMAP) and update the issue? If so, nothing is happening.
Our settings in the /etc/gitlab/gitlab.rb (and I had to add the "incoming-mail" section manually because it was not there) looks like this:
# SMTP setup
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "aws"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "AWSUSER"
gitlab_rails['smtp_password'] = "AWSPASS"
gitlab_rails['smtp_domain'] = "git.ourdomain.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
# gitlab_rails['smtp_tls'] = false
# gitlab_rails['smtp_openssl_verify_mode'] = 'none' # Can be: 'none', 'peer', 'client_once', 'fail_if_no_peer_cert', see http://api.rubyonrails.org/classes/ActionMailer/Base.html
# gitlab_rails['smtp_ca_path'] = "/etc/ssl/certs"
# gitlab_rails['smtp_ca_file'] = "/etc/ssl/certs/ca-certificates.crt"
# Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming#gmail.com
gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+%{key}#ourdomain.com"
gitlab_rails['incoming_email_email'] = "gitlab#ourdomain.com"
gitlab_rails['incoming_email_password'] = "GLPASS"
gitlab_rails['incoming_email_host'] = "imap.gmail.com"
gitlab_rails['incoming_email_port'] = 993
gitlab_rails['incoming_email_ssl'] = true
gitlab_rails['incoming_email_start_tls'] = false
gitlab_rails['incoming_email_mailbox_name'] = "inbox"
For me installing the last update and restarting the server seemed to solve the problem (I did restart the server the first time as well but it still was not working).
I've got some strange behavior between me running a command under my profile, and TeamCity running the same command.
> Import-Module $root\packages\fsm.buildrelease.*\tools\modules\BuildDeployModules -Force
If I run the script manually, the build kicks off as expected. If I let TeamCity execute the script, it pukes with the following.
Import-Module : The specified module 'C:\BuildAgent\work\81eb7c2fdfcfc0af\packages\fsm.buildrelease.*\tools\modules\BuildDeployModules' was not loaded because no valid module file was found in any module directory.
I have double and triple verified that the module exists in that location. And I've gone through my modules and added [cmdletbinding()] before the param, but it doesn't seem to solve this issue.
It's frustrating because it doesn't say "which" module is getting an invalid parameter passed in.
$fsmbrVersion = "1.1.1" # contains the current version of fsm.buildrelease
Write-Host "`nfsm.buildrelease version $fsmbrVersion `nCopyright ($([char]0x00A9)) Future State Mobile Inc. & Contributors`n"
Push-Location $psScriptRoot
. .\Add-HostsFileEntry.ps1
. .\Add-IISHttpVerb.ps1
. .\Add-IISMimeType.ps1
. .\Add-LoopbackFix.ps1
. .\ApplicationAdministration.ps1
. .\AppPoolAdministration.ps1
. .\Approve-Permissions.ps1
. .\Assert-PSVersion.ps1
. .\EntityFramework.ps1
. .\Expand-NugetPackage.ps1
. .\Expand-ZipFile.ps1
. .\Format-TaskNameToHost.ps1
. .\Get-EnvironmentSettings.ps1
. .\Grunt.ps1
. .\Helpers.ps1
. .\Install-WebApplication.ps1
. .\Invoke-Deployment.ps1
. .\Invoke-DeployOctopusNugetPackage.ps1
. .\Invoke-ElevatedCommand.ps1
. .\Invoke-ExternalCommand.ps1
. .\Invoke-Using.ps1
. .\MSBuild.ps1
. .\Nuget.ps1
. .\nUnit.ps1
. .\Set-IISAuthentication.ps1
. .\Set-IISCustomHeader.ps1
. .\SiteAdministration.ps1
. .\Specflow.ps1
. .\SqlHelpers.ps1
. .\Test-PathExtended.ps1
. .\Test-RunAsAdmin.ps1
. .\TextUtils.ps1
. .\Update-AssemblyVersions.ps1
. .\Update-JsonConfigFile.ps1
. .\Update-XmlConfigFile.ps1
. .\WindowsFeatures.ps1
. .\xUnit.ps1
Pop-Location
Export-ModuleMember `
-Alias #(
'*') `
-Function #(
'Add-HostsFileEntry',
'Add-IISHttpVerb',
'Add-IISMimeType',
'Add-LoopbackFix',
'Approve-Permissions',
'Assert-That',
'Assert-PSVersion',
'Confirm-ApplicationExists',
'Confirm-AppPoolExists',
'Confirm-SiteExists',
'Exec',
'Expand-NugetPackage',
'Expand-ZipFile',
'Format-TaskNameToHost',
'Get-Application',
'Get-Applications',
'Get-AppPool',
'Get-AppPools',
'Get-DatabaseConnection',
'Get-EnvironmentSettings',
'Get-Site',
'Get-Sites',
'Get-TestFileName',
'Get-WarningsFromMSBuildLog',
'Get-WindowsFeatures',
'Install-WebApplication',
'Install-WindowsFeatures',
'Invoke-BulkCopy',
'Invoke-DBMigration',
'Invoke-Deployment',
'Invoke-DeployOctopusNugetPackage',
'Invoke-ElevatedCommand',
'Invoke-EntityFrameworkMigrations',
'Invoke-ExternalCommand',
'Invoke-FromBase64',
'Invoke-GruntMinification',
'Invoke-HtmlDecode',
'Invoke-HtmlEncode',
'Invoke-KarmaTests',
'Invoke-MSBuild',
'Invoke-Nunit',
'Invoke-NUnitWithCoverage'
'Invoke-SpecFlow',
'Invoke-SqlFile',
'Invoke-SqlStatement',
'Invoke-ToBase64',
'Invoke-UrlDecode',
'Invoke-UrlEncode',
'Invoke-Using',
'Invoke-XUnit',
'Invoke-XUnitWithCoverage',
'New-Application',
'New-AppPool',
'New-NugetPackage',
'New-Site',
'Remove-Application',
'Remove-AppPool',
'Remove-Site',
'RequiredFeatures',
'Set-IISAuthentication',
'Set-IISCustomHeader',
'Start-Application',
'Start-AppPool',
'Start-Site',
'Step',
'Stop-Application',
'Stop-AppPool',
'Stop-Site',
'Test-PathExtended',
'Test-RunAsAdmin',
'Update-Application',
'Update-AppPool',
'Update-AssemblyVersions',
'Update-JsonConfigValues',
'Update-Site',
'Update-XmlConfigValues'
)
# Messages
DATA msgs {
convertfrom-stringdata #"
error_duplicate_step_name = Step {0} has already been defined.
error_must_supply_a_feature = You must supply at least one Windows Feature.
error_feature_set_invalid = The argument `"{0}`" does not belong to the set `"{1}`".
error_admin_required = You are required to 'Run as Administrator' when running this deployment.
error_loading_sql_file = Error loading '{0}'. {1}.
error_octopus_deploy_failed = Failed to deploy: {0}.
error_specflow_failed = Publishing specflow results for '{0}' failed.
error_coverage_failed = Running code coverage for '{0}' failed.
error_tests_failed = Running tests '{0}' failed.
error_msbuild_compile = Error compiling '{0}'.
wrn_full_permission = You have applied FULL permission to '{0}' for '{1}'. THIS IS DANGEROUS!
wrn_cant_find = Could not find {0} with the name: {0}.
msg_grant_permission = Granting {0} permissions to {1} for {2}.
msg_enabling_windows_feature = Enabling Windows Feature: `"{0}`".
msg_wasnt_found = `"{0}`" wasn't found.
msg_updated_to = Updated `"{0}`" to `"{1}`".
msg_updating_to = Updating `"{0}`" to `"{1}`".
msg_changing_to = Changing `"{0}`" to `"{1}`".
msg_overriding_to = Overriding node `"{0}`" with value `"{1}`".
msg_updating_assembly = Updating AssemblyVersion to '{0}'. Updating AssemblyFileVersion to '{1}'. Updating AssemblyInformationalVersion to '{2}'.
msg_not_updating = Not updating {0}, you must specify the '-updateIfFound' if you wish to update the {0} settings.
msg_custom_header = Setting custom header '{0}' on site '{1}' to value '{2}'.
msg_disable_anon_auth = Disabling Anonymous Authentication for '{0}'.
msg_web_app_success = Successfully deploy Web Application '{0}'.
msg_copying_content = Copying {0} content to {1}.
msg_use_machine_environment = Using config for machine {0} instead of the {1} environment.
msg_octopus_overrides = Checking for Octopus Overrides for environment '{0}'.
msg_teamcity_importdata = ##teamcity[importData type='{0}' tool='{1}' path='{2}']
msg_teamcity_buildstatus = ##teamcity[buildStatus text='{0}']
msg_teamcity_buildstatisticvalue = ##teamcity[buildStatisticValue key='{0}' value='{1}']
msg_add_loopback_fix = Adding loopback fix for '{0}'.
msg_add_mime_type = Adding mime type '{0}' for extension '{1}' to IIS site '{2}'.
msg_add_verb = Adding IIS Http Verb '{0}' to site '{1}'.
msg_add_host_entry = Adding host entry for '{0}' into the hosts file.
msg_validate_host_entry = Validating host entry for '{0} in the hosts file'
msg_loopback_note = note: we're not disabling the loopback check all together, we are simply adding '{0}' to an allowed list.
"#
}
The problem is nothing to do with importing the module; powershell isn't very helpful in leading you to the issue. The root of the issue didn't reveal it's self until I put the -VERBOSE switch on the import. Once I did that, I got a new error above the old one.
: Parameter attributes need to be a constant or a script block.
FullyQualifiedErrorId: ParameterAttributeArgumentNeedsToBeConstandOrScriptBlock
Essentially, I was using ValidatePattern with double quotes instead of single quotes. Check and make sure your writing your regex patterns correctly when you use ValidatePattern
# Bad
[ValidatePattern("^[a-z]$")]
# Good
[ValidatePattern('^[a-z]$')]
I've a TFS Build Definition setup where, I execute a powershell script for security hardening, applying final patches etc.
Sometimes this powershell script can fail and I would like to change the status of build to be un-successfull or fail. Is there a way to change this TFS Build status via a powershell?
Try this:
$script:TFServerName = $ServerName
$script:TFServer = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($ServerName)
$script:TFBuildServer = $TFServer.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $TFBuildServer.CreateBuildDetailSpec($TeamProject, $DefinitionName)
$spec.Status = 'Failed'
Make sure you've Add-Type the right versions of the TFS assemblies. I'm assuming here that since the properties support setters, they communicate back to the server to affect the change. But I'm not 100% sure of that.
Here's what I pieced together from MSDN:
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
$tfsServerAddress = "http://{tfs hostname}:{port}/{CollectionName}"
$build = "{build definition name}_{build date}.{build count for the day}"
#update build status
$tfsProject = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($tfsServerAddress)
$tfsBuildServer = $tfsProject.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $tfsBuildServer.CreateBuildDetailSpec($teamProject)
$spec.BuildNumber = $build
$builds = $tfsBuildServer.QueryBuilds($spec).Builds
if ($builds[0].TestStatus -eq 'Succeeded'){
echo "updating test status:" $builds[0].TestStatus "to $testOutcome"
$builds[0].TestStatus = $testOutcome
}
if ($builds[0].Status -eq 'Succeeded'){
echo "updating build status:" $builds[0].Status "to $testOutcome"
$builds[0].Status = $testOutcome
}
$tfsBuildServer.SaveBuilds($builds)
Keep in mind that you need a special permission to update build status
I'm trying to automate the deployment process, and as part of it, I need to run my release build from command line. I can do it, using command like
.\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
It even returns some code for the queued build — Build queued. Queue position: 2, Queue ID: 11057.
What I don't know, is how to get info about currently running builds, or about the state of my running build from powershell command line? The final aim is to start publishing after that build completes.
I've already got all necessary powershell scripts to create the deployment package from the build results, zip it, copy to production and install there. All I need now — to know when my build succeedes.
This function will wait for a build with the Queue ID given by TFSBuild.exe:
function Wait-QueuedBuild {
param(
$QueueID
)
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Build.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Client')
$uri = [URI]"http://server-name:8080/tfs/project-collection"
$projectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($uri)
$buildServer = $projectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $buildServer.CreateBuildQueueSpec('*','*')
do {
$build = $buildServer.QueryQueuedBuilds($spec).QueuedBuilds| where {$_.Id -eq $QueueID}
sleep 1
} while ($build)
}
You can get the id returned by TFSBuild.exe, then call the function.
$tfsBuild = .\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
Wait-QueuedBuild [regex]::Match($tfsBuild[-1],'Queue ID: (?<id>\d+)').Groups['id'].Value
Using the work by E.Hofman available here it is possible to write a C# console app that uses TFS SDK and reveals if any build agent is currently running as follows:
using System;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
namespace ListAgentStatus
{
class Program
{
static void Main()
{
TfsTeamProjectCollection teamProjectCollection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://TFSServer:8080"));
var buildServer = teamProjectCollection.GetService<IBuildServer>();
foreach (IBuildController controller in buildServer.QueryBuildControllers(true))
{
foreach (IBuildAgent agent in controller.Agents)
{
Console.WriteLine(agent.Name+" is "+agent.IsReserved);
}
}
}
}
}
The parameter .IsReserved is what toggles to 'True' during execution of a build.
I 'm sorry my powershell skills are not good enough for providing with a PS variant of the above. Please take a look here, where the work by bwerks might help you do that.
# load classes for execution
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client") | Out-Null
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client") | Out-Null
# declare working variables
$Uri = New-Object System.Uri "http://example:8080/tfs"
# get reference to projection collection
$ProjectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($Uri)
# get reference to build server
$BuildServer = $ProjectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
# loop through the build servers
foreach($Controller in $BuildServer.QueryBuildControllers($true))
{
# loop through agents
foreach($BuildAgent in $Controller.Agents)
{
Write-Host "$($BuildAgent.Name) is $($BuildAgent.IsReserved)"
}
}