How to deploy with Release Management to remote datacenter - deployment

We are running TFS and Release Management on premises, and i want to deploy my applications to a remote datacenter.
Access is over the internet, so there is no windows shares available.
I am using the vNext templates, and afaik RM seems to only support unc paths over windows shares.
How can i use Release Management to deploy software to this datacenter?
Im working on this solution:
Use WebDav on a IIS located inside the datacenter.
RM server and Target can use the WebDav client built into windows and access it by an unc path.
I haven't gotten this to work yet, as RM won't use the correct credentials to logon to the webdav server.
Updated with my solution
This is only a proof of concept, and is not production tested.
Setup a WebDav site accessible from both RM server and Target server
Install the feature "Desktop experience" on both servers
Make the following DLL
using System;
using System.ComponentModel.Composition;
using System.Diagnostics;
using System.IO;
using Microsoft.TeamFoundation.Release.Common.Helpers;
using Microsoft.TeamFoundation.Release.Composition.Definitions;
using Microsoft.TeamFoundation.Release.Composition.Services;
namespace DoTheNetUse
{
[PartCreationPolicy(CreationPolicy.Shared)]
[Export(typeof(IThreadSafeService))]
public class DoTheNetUse : BaseThreadSafeService
{
public DoTheNetUse() : base("DoTheNetUse")
{}
protected override void DoAction()
{
Logger.WriteInformation("DoAction: [DoTheNetUse]");
try
{
Logger.WriteInformation("# DoTheNetUse.Start #");
Logger.WriteInformation("{0}, {1}", Environment.UserDomainName, Environment.UserName);
{
Logger.WriteInformation("Net use std");
var si = new ProcessStartInfo("cmd.exe", #"/c ""net use \\sharedwebdavserver.somewhere\DavWWWRoot\ /user:webdavuser webdavuserpassword""");
si.UseShellExecute = false;
si.RedirectStandardOutput = true;
si.RedirectStandardError = true;
var p = Process.Start(si);
p.WaitForExit();
Logger.WriteInformation("Net use output std:" + p.StandardOutput.ReadToEnd());
Logger.WriteInformation("Net use output err:" + p.StandardError.ReadToEnd());
}
//##########################################################
Logger.WriteInformation("# Done #");
}
catch (Exception e)
{
Logger.WriteError(e);
}
}
}
}
Name it "ReleaseManagementMonitor2.dll"
Place it in the a subfolder to The service "ReleaseManagementMonitor"
Configure the shared path as the solution below states.
DO NOT OVERWITE THE EXISTING "ReleaseManagementMonitor2.dll"
The reason that this works is MEF.
The ReleaseManagementMonitor service tries to load the dll "ReleaseManagementMonitor2.dll" from all subfolders.
This dll implements a service interface that RM recognises.
It the runs "net use" to apply the credentials to the session that the service runs under, and thereby grants access to the otherwise inaccessible webdav server.
This solution is certified "Works on my machine"

RM does work only with UNC, you are right on that.
You can leverage that to make your scenario work -
In Theory
Create a boundary machine on the RM domain, where your drops can be copied.
The deploy action running on your datacenter can then copy bits from this boundary machine, using credentials that have access on that domain. (These credentials are provided by you in the WPF console)
How this works
1. Have a dedicated machine on the RM server domain (say D1) that will be used as a boundary machine.
2. Define this machine as a boundary machine in RM by specifying a shared path that will be used by your data centre. Go to settings tab in your WPF console, create a new variable - { Key = RMSharedUNCPath, Value = \\BoundaryMachine\DropsLocation }. RM now understands you want to use this machine as your boundary machine.
3. Make sure you take care of these permissions
RM Server should have write permissions on the \\BoundaryMachine\DropsLocation share.
Pass down credentials of domain D1 to the target machine in the data centre (Domain D2), that can be used to access the share.
4. Credentials can be passed down fron the WPF console, you will have to define the following two config variables in the settings tab again.
Key = RMSharedUNCPathUser ; Value = domain D1 user name
Key = RMSharedUNCPathPwd ; Value = password for the user defined above.
PS - Variable names are case sensitive.
Also, to let RM know that you want to use the SharedUNC mechanism, check the corresponding checkbox for the RM server and connect to it via IP and not DNS name as these must be in different domains, i.e.

Try to use Get-Content on local-server then Set-Content on the remote server passing the file contents over;
Could package everything into an archive of some kind.

The Release Management is copying VisualStudioRemoteDeployer.exe to C:\Windows\DtlDownloads\VisualStudioRemoteDeployer folder on the target server then is copying the scripts from the specified location to target server using robocopy.
So you have to give permissions from your target server to your scripts location.

Release Management update 4 supports "Build drops stored on TFS servers"
http://blogs.msdn.com/b/visualstudioalm/archive/2014/11/11/what-s-new-in-release-management-for-vs-2013-update-4.aspx

Related

What is the "[full path]" component of the SSL Certificate Authority given by MySQL and PostgreSQL (boto3) calls in the AWS docs?

In the AWS documentation for "Connecting to your DB instance using IAM authentication and the AWS SDK for Python (Boto3)", the following call is made to both psycopg2.connect (shown) and mysql.connector.connect:
conn = psycopg2.connect(host=ENDPOINT, port=PORT, database=DBNAME, user=USR, password=token, sslmode='prefer', sslrootcert="[full path]rds-combined-ca-bundle.pem")
cur = conn.cursor()
cur.execute("""SELECT now()""")
query_results = cur.fetchall()
print(query_results)
I see some discussion about the ssl_ca path (here and here) and what those bundles are used for. But none of the three links I've given here describe the [full path] component given by the AWS docs, or where it is pointing to. My current guess (from the second link) is this URL, but I'd like to be sure.
Additionally, what are the advantages to having this bundle downloaded to the remote EC2 on which these Python 3 (boto3) scripts are running?
EDIT: By the way, the above call to psycopg2.connect is working in Jupyter with Python 3.9.5 on an EC2 currently, with the [full path] written as-is...
You should replace the '[full path]' with the filesystem path (directory path) to where you saved the pem file when you downloaded it (from that last URL you gave) to the local computer.
The advantage of using it is that your client will verify it connected to the correct database, and not some malicious system which is intercepting your traffic. I don't how advantageous you consider this: if someone has compromised Amazon enough to be intercepting their internal traffic, they might also have compromised their CA as well. But there is at least some possibility they did one without the other.
Your code as shown does not work for me, because ssl_ca is not how it is spelled. Assuming you used the code actually given at your first link for PostgreSQL:
sslmode='prefer', sslrootcert="[full path]rds-combined-ca-bundle.pem"
Then the reason it works despite the bogus path is that "prefer" means it doesn't care if the rootcert is missing, it just skips validating in that case. If you change it to 'verify-full', then presumably it would stop working.

How to setup a machine as a mirror server for Yocto when fetching packages?

When building a project with Petalinux (a type of Yocto), it needs Internet for fetching packages from server (git serve or others).
My working machine does not have permission for accessing Internet (just only have LAN), so I have a plan to set up a machine in this LAN that can access to Internet to become a mirror server for Yocto.
Does anyone have any idea for setting up a server like this? Please help.
You can check the following pages to setup a source mirror:
Source download mirror
Setting up mirrors
Replicating a build offline
Basically, you launch a build on source mirror machine with those options:
SOURCE_MIRROR_URL ?= "file:///source_mirror/sources/"
INHERIT += "own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "1"
You can only fetch source with following command:
bitbake -c target runall="fetch".
Then you launch an ftp server that serves ./source_mirror/sources/ folder on http://example.com/my-source-mirror.
Then on offline machine, you set
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "http://example.com/my-source-mirror"
BB_NO_NETWORK = "1" # or BB_FETCH_PREMIRRORONLY = "1"
If you have access to a proxy you can check those:
sources behind proxy
working behind proxy
Copy'n'paste shortcut: Below’s a working configuration you can just copy’n’paste without investing the time to understand every little detail :)
Architecture: In this example there are two types of machine. The "build server" and several instances of "developer pc“.
Machine Preparation:
Install a shared folder on all machines (server and developer) giving access to any kind of file server (e.g. nfs) mapping its storage to /mnt/mirror.
Example for NFS in case this is new for you, skip if you have NFS already mounted: https://pelux.io/2017/06/19/How-to-create-a-shared-sstate-dir.html (Stop reading at the caption „Yocto“ and proceed as below)
Overall Configuration:
Add the code I pasted to the end of this ticket to the file conf/local.conf and remove all prior lines that conflict (i.e. mess with any of the variables we defined like DL_DIR)
Machine configuration:
For the developer machines use A (outcomment B) and for the build server use B (outcomment A).
Hit it:
When the server PC bitbakes for the first time it populates the mirror folders. After the first server build finished the clients will use the mirror. (source-mirror to bypass internet dependencies and sstate-cache to speed up build speed).
local.conf:
# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# Activate either A or B depending on it this is a developer pc or the build server
# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
MIRROR_SERVER = "file:///mnt/mirror/"
# ########################################################
# A) Settings for developer PC operation
# ########################################################
BB_FETCH_PREMIRRORONLY = "1"
SOURCE_MIRROR_URL = "${MIRROR_SERVER}/source-mirror"
UNINATIVE_URL = "${SOURCE_MIRROR_URL}"
INHERIT += "own-mirrors"
SSTATE_MIRRORS = "\
file://.* ${MIRROR_SERVER}/sstate-cache/PATH;\
downloadfilename=PATH \n \
"
# ########################################################
# B) SETTINGS FOR BUILDSERVER OPERATION
# ########################################################
#SSTATE_DIR = "/mnt/remux/sstate-cache"
#BB_GENERATE_MIRROR_TARBALLS = "1"
##To populate the source mirror start a normal server build or run: bitbake --runall=fetch <image>
# ########################################################
# SETTINGS FOR BOTH, A and B
# ########################################################
DL_DIR = "/mnt/mirror/source-mirror“

AWS MobileAnalyticsManager access to folder 'AWS Mobile Services\M4SP' is denied

I am trying to add the AWSSDK DLL into my C# code to collect my event data and pass the data to the AWS bucket. My C# code is created with VS Share point template. The project contains WSP files. The following code indicates how I use the AWSSDK :
using Amazon;
using Amazon.CognitoIdentity;
using Amazon.MobileAnalytics.MobileAnalyticsManager;
CognitoAWSCredentials credentials = new CognitoAWSCredentials(
"us-east-1:xxxxxx",//PoolID
RegionEndpoint.USEast1
);
Amazon.AWSConfigs.ApplicationName = "M4SP";
AWSConfigs.LoggingConfig.LogMetrics = true;
AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
AWSConfigs.LoggingConfig.LogMetricsFormat = LogMetricsFormatOption.JSON;
MobileAnalyticsManager manager = MobileAnalyticsManager.GetOrCreateInstance(
"xxxxxxxxxxxxxxxxxxx",//AppID
credentials,
RegionEndpoint.USEast1 // Region
);
CustomEvent customEvent = new CustomEvent("TestRecordEvent");
customEvent.AddAttribute("label", "M4SP");
customEvent.AddAttribute("action", "invoke");
customEvent.AddAttribute("details", "run the workflow test");
manager.RecordEvent(customEvent);
I found the code inside AWSSDK DLL was trying to log the data to local folder before passing it to AWS database. The location of the folder is C:\Users\[userid]\AppData\Roaming\AWS Mobile Services.
There is no problem in a standalone project since it always uses current user’s identity to run the application so it has access to the folder. But, because of the authentication mechanism of SharePoint solutions, it uses Application Pool Identity to access the folder and it gets access denied issue and the whole process fails.
Here is the error:
"Access to the path 'AWS Mobile Services\M4SP' is denied."
I modified the access right of Share point Application Pool Identity (in my case, it is “network service” account) but it still can’t access the folder .
Does anyone have a solution for this issue? Thanks very much for the help!!

Entity Framework Code First ignoring connection string, using IIS instead

I have a web app that I've created using Entity Framework Code First. In setting it up I have managed to match my DB connection string to my DBContext by specifying the full namespace and class of the DBContext as the name of the connection string.
<add name="MyClassProject.EfDbContext" connectionString="Data Source=.\SQLEXPRESS;Initial Catalog=MyDatabase;Integrated Security=true;User Id=MyUsername;Password=MyPassword;" providerName="System.Data.SqlClient"/>
Initially when I set up the project, I just had it created in c:\inetpub\wwwroot, and just ran it through Visual Studio. Everything worked fine.
Now I'm trying to have the code build to a separate website folder, and have the website run as it's own website and app pool in IIS. I've set up the website, and my hosts file but when I went to run it I received the following error.
Cannot open database "MyDatabase" requested by the login. The login failed.
Login failed for user 'IIS APPPOOL\MyAppPool'.
I'm wondering why this is happening, as I seem to be specifying the security username and password to use for the DB in my connection string....so why is it trying to connect as the app pool that my website is running in?
Also, how can I fix this, without having to give MyAppPool (or Network Service if I changed it to that) DB permissions in SQL Server?
Update: I should've mentioned that I initialise my DBContext class using:
namespace MyClassProject
{
public class EfDbContext : DbContext
{
public EfDbContext() : base ("MyDatabase")
{
}
}
}
I found the issue.
When I initialise my DBContext class with : base("MyDatabase"), it overrides the connection string specified in the web.config.
Removing that from my DBContext class, with the database already existing, the site now works in IIS.
However, if I don't have the database created already, (or if I have my database initialiser use DropCreateDatabaseWhenModelChanges or DropCreateDatabaseAlways so that it'll needs to recreate the DB), the initialiser will fail, as it'll try to use an SQL user that doesn't have permissions to create the DB.
My way around it is to use the : base("MyDatabase") and run from Visual Studio initially so the database is created.
Then remove it from code, add the specified user to the DB security in SQL Server, and it'll allow my site to run in IIS thereafter.
Remove Integrated Security=true;. That is the setting that passes the current user off.
When using Integrated Security, the DB is given a token from the user who is currently running the process. In all likelihood, you run Visual Studio from your user account, which likely has Admin permissions on your SQL Server instance.
When IIS runs your app, it uses something called an Application Pool (or App pool). You can have multiple apps in a single pool to be managed together. The app pool also runs under a special user account named for the pool. App pool users exist under a container called "IIS AppPool", so the local user for the DefaultAppPool is IIS AppPool\DefaultAppPool. If you want to grant access to a resource on your local system (including file permissions), you can also grant it to the app pool user or local group IIS_IUSRS to grant it to all app pools.
Remember that these are local accounts, so they will not cross network boundaries. To grant permissions on a different server, you'll need to either use a domain user (or even better, a domain Managed Service Account) or you can set the app pool user to NETWORK SERVICE and then you can grant permissions to MyDomain\MyWebServer$ (the dollar sign is important).
You can use Web.config Transform to have Local connection stirng different from Remote (say in Release mode). To start using it you need to publish your Web App using One-Click Publish from Visual Studio. That's really very handy way to publish web apps!
Looks like that's what you're looking for.
Or set connection string name base on a condition:
public EfDbContext() : base (GetConnectionStringName())
{
}
private static GetConnectionStringName()
{
return RunLocally() : "LocalDatabase" : "RemoteDatabase";
}
private static bool RunLocally()
{
// implement some how
}

Enterprise Library for Logging to Flat File Trace Listener

What kind of permission needs to be given to the Error.log file in the server (running IIS 7.5) so that errors are written to the Error.log file?
I have created a directory named ErrorLog and set up the listener in web.config like below.
The windows identity in play (depends on how you authenticate)
var principal = ClaimsPrincipal.Current; //normally this reverts to Thread.CurrentPrincipal, but can be changed
return principal.Identity.Name;
or
var windowsIdentity = WindowsIdentity.GetCurrent();
if (windowsIdentity != null)
{
return windowsIdentity.Name;
}
The Executing windows identity Will need create permission in directory and write permission to the file specified in your EL config.
You can place file anywhere you like using config. Default for file without path is Start project directory containing assembly dll.
See In IIS the Application Pool used by the website. The pool determines the windows Identity. See the advanced settings tab for the Pool. The identity approach to be used is defined here.
I don't see your configuration example, but..
I find that it is best not to use Flat File outside of development. The Event Log or Database listeners are more suitable for a web application in production.
If you must use Flat File, you will need to give the account (for the application pool that is assigned to the web application) write access to the directory that the file should be created in. By default I think that is the web application's bin directory, unless you include a path in the file name in your configuration.
It probably depends on what context your app in operating in. The app pool thread is responsible for writing the log so give the read/write folder permissions for the same identity as your app pool. I think this is NETWORKSERVICE by default.