How can I alter a Chocolatey package's $url for install - powershell

TLDR version: I need to be able to append to the end of the URL that a chocolatey package specifies for its download source.
I'm trying to use Chocolatey to install the Android-SDK package and am having an issue with my filtering proxy (the functionality and provision of which, I have no control over). The proxy is detected properly but it will block URLs that contain certain patterns and issue a 403 return (in this case it's because the URL is to a .exe). So running choco install -y android-sdk causes the following error.
2015-03-12 15:14:47,639 [INFO ] - ============================================================
2015-03-12 15:14:47,664 [INFO ] - Chocolatey v0.9.9.2
2015-03-12 15:14:47,730 [INFO ] - Installing the following packages:
2015-03-12 15:14:47,753 [INFO ] - android-sdk
2015-03-12 15:14:47,754 [INFO ] - By installing you accept licenses for the packages.
2015-03-12 15:14:49,221 [INFO ] -
android-sdk v24.0.2
2015-03-12 15:14:50,525 [INFO ] - Using this proxyserver: xxx.xxx.xxx.xxx:xxxxx
2015-03-12 15:14:52,175 [INFO ] - Attempt to get headers for http://dl.google.com/android/installer_r24.0.2-windows.exe failed.
2015-03-12 15:14:52,175 [INFO ] - Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (403) Forbidden."
2015-03-12 15:14:52,180 [INFO ] - Downloading android-sdk 32 bit
2015-03-12 15:14:52,180 [INFO ] - from 'http://dl.google.com/android/installer_r24.0.2-windows.exe'
2015-03-12 15:14:52,213 [INFO ] - Using this proxyserver: xxx.xxx.xxx.xxx:xxxxx
2015-03-12 15:14:52,329 [ERROR] - Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (403) Forbidden."
2015-03-12 15:14:52,330 [ERROR] - At C:\ProgramData\chocolatey\helpers\functions\Get-WebFile.ps1:66 char:3
2015-03-12 15:14:52,330 [ERROR] - + $res = $req.GetResponse();
2015-03-12 15:14:52,330 [ERROR] - + ~~~~~~~~~~~~~~~~~~~~~~~~~
2015-03-12 15:14:52,331 [ERROR] - + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
2015-03-12 15:14:52,331 [ERROR] - + FullyQualifiedErrorId : WebException
2015-03-12 15:14:52,354 [ERROR] - You cannot call a method on a null-valued expression.
2015-03-12 15:14:52,354 [ERROR] - At C:\ProgramData\chocolatey\helpers\functions\Get-WebFile.ps1:138 char:3
2015-03-12 15:14:52,355 [ERROR] - + $res.Close();
2015-03-12 15:14:52,356 [ERROR] - + ~~~~~~~~~~~~
2015-03-12 15:14:52,356 [ERROR] - + CategoryInfo : InvalidOperation: (:) [], RuntimeException
2015-03-12 15:14:52,357 [ERROR] - + FullyQualifiedErrorId : InvokeMethodOnNull
2015-03-12 15:14:54,374 [ERROR] - Chocolatey expected a file to be downloaded to
2015-03-12 15:14:54,375 [ERROR] - 'X:\Users\xxx\AppData\Local\Temp\chocolatey\android-sdk\android-sdkInstall.exe' but nothing exists at that
2015-03-12 15:14:54,375 [ERROR] - location.
2015-03-12 15:14:54,376 [ERROR] - At C:\ProgramData\chocolatey\helpers\functions\Get-ChocolateyWebFile.ps1:148 char:24
2015-03-12 15:14:54,376 [ERROR] - + if (!($fi.Exists)) {{ throw "Chocolatey expected a file to be downloaded to `'$ ...
2015-03-12 15:14:54,377 [ERROR] - + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2015-03-12 15:14:54,377 [ERROR] - + CategoryInfo : OperationStopped: (Chocolatey expe... that location.:String) [], RuntimeException
2015-03-12 15:14:54,378 [ERROR] - + FullyQualifiedErrorId : Chocolatey expected a file to be downloaded to 'X:\Users\xxx\AppData\Local\Temp\cho
2015-03-12 15:14:54,378 [ERROR] - colatey\android-sdk\android-sdkInstall.exe' but nothing exists at that location.
2015-03-12 15:14:54,618 [ERROR] - android-sdk install not successful.
2015-03-12 15:14:54,621 [ERROR] - Error while running 'C:\ProgramData\chocolatey\lib\android-sdk\tools\chocolateyInstall.ps1'.
See log for details.
2015-03-12 15:14:56,643 [WARN ] -
Chocolatey installed 0/1 package(s). 1 package(s) failed.
See the log for details.
2015-03-12 15:14:56,643 [ERROR] - Failures:
2015-03-12 15:14:56,645 [ERROR] - - android-sdk
I could create my own package with the chocolateyInstall.ps1 and chocolateyUninstall.ps1 listed on the package's web page and alter the URL, but that negates any benefit of using a package manager to simplify the process of managing programs and dependencies.
So I am wondering if there is some way that I can simply append a string that allows the URL through the proxy regardless of the file extension (to be clear, I know what this string is, I just don't know how to append it to the URL specified by the package).

This isn't a complete answer and I'm not sure how safe this answer is (in terms of surviving updates to Chocolatey itself), but it's resolved my immediate issue with the android-sdk package. Unfortunately, any package that implements its own download code bypasses this "fix".
I noticed the line in the error log that highlights one of the failure points as 2015-03-12 15:14:52,330 [ERROR] - At C:\ProgramData\chocolatey\helpers\functions\Get-WebFile.ps1:66 char:3 and wondered if I could alter the URL in that powershell script. After a bit of trial and error with the exact escaping that was needed for the appended string I managed to get it downloading the file, but the header check was still failing.
I then checked the other files in the helpers/functions/ directory and found Get-ChocolateyWebFile.ps1 which is what appears to call Get-WebFile.ps1 and also does the header check to see if it needs SSL.
So, I created a new powershell function file to perform the necessary URL rewrites in a function named Get-ProxyUrl and added this line to Get-ChocolateyWebFile.ps1 after the section that checks for 32bit/64bit URL but before the header check on the URL (line 101 in my version 0.9.9.2 file).
$url = Get-ProxyUrl($url)
I then also added it to the very beginning of the Get-WebFile function definition in Get-WebFile.ps1 to capture any direct calls to that function.
So, this will allow me to fix any packages that use Install-ChocolateyPackage, Get-ChocolateyWebFile or Get-WebFile, but still leaves some packages - such as JDK8 - that specify their own file download code and still fail.
So, a better solution would capture any URLs from a Chocolatey install... probably the only way to do this would be to install a local proxy server that chocolatey can use that would transform the URL before being passed on

Related

Azure file copy to blob failing

Trying the Azure file copy for the first time so not sure where I'm going wrong as it keeps failing with the details seen further below. It does create the 'Test' container though.
Also tried by changing the task version to 2 and 3. Same result
Any suggestions where to start looking?
Copied from 'View Yaml'
steps:
- task: AzureFileCopy#4
displayName: 'AzureBlob File Copy'
inputs:
SourcePath: '$(System.DefaultWorkingDirectory)/interace/interface brands/v20.08'
azureSubscription: Development
Destination: AzureBlob
storage: updates
ContainerName: Test
Log output:
2020-08-20T12:51:39.0807389Z INFO: AzCopy.exe: A newer version 10.6.0 is available to download
2020-08-20T12:51:39.0810239Z
2020-08-20T12:51:39.0923021Z ##[debug]Trying to disconnect from Azure and clear context at process scope
2020-08-20T12:51:39.0975253Z ##[debug]Cannot verify the Microsoft .NET Framework version 4.7.2 because it is not included in the list of permitted versions.
2020-08-20T12:51:39.0997871Z ##[debug]Populating RepositorySourceLocation property for module Az.Accounts.
2020-08-20T12:51:39.1022385Z ##[debug]Loading module from path 'C:\Modules\az_3.1.0\Az.Accounts\1.9.2\Az.Accounts.psm1'.
2020-08-20T12:51:39.1250145Z ##[command]Disconnect-AzAccount -Scope Process -ErrorAction Stop
2020-08-20T12:51:39.5095303Z ##[command]Clear-AzContext -Scope Process -ErrorAction Stop
2020-08-20T12:51:40.0741695Z ##[debug]Caught exception from task script.
2020-08-20T12:51:40.0773655Z ##[debug]Error record:
2020-08-20T12:51:40.1256747Z ##[debug]Upload to container: 'test' in storage account: 'updates' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-08-20T12:51:40.1267682Z ##[debug]At D:\a\_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\4.171.3\Utility.ps1:109 char:5
2020-08-20T12:51:40.1278911Z ##[debug]+ throw "$errorMessage $helpMessage"
2020-08-20T12:51:40.1290483Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-08-20T12:51:40.1301825Z ##[debug] + CategoryInfo : OperationStopped: (Upload to conta...efilecopyreadme:String) [], RuntimeException
2020-08-20T12:51:40.1313423Z ##[debug] + FullyQualifiedErrorId : Upload to container: 'test' in storage account: 'updates' with blob prefix: '' fa iled with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-08-20T12:51:40.1323723Z ##[debug]
2020-08-20T12:51:40.1343144Z ##[debug]Script stack trace:
2020-08-20T12:51:40.1387327Z ##[debug]at ThrowError, D:\a\_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\4.171.3\Utility.ps1: line 109
2020-08-20T12:51:40.1399427Z ##[debug]at Upload-FilesToAzureContainer, D:\a\_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\4.171.3\Utility.ps1: line 248
2020-08-20T12:51:40.1410626Z ##[debug]at <ScriptBlock>, D:\a\_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\4.171.3\AzureFileCopy.ps1: line 181
2020-08-20T12:51:40.1421592Z ##[debug]at <ScriptBlock>, <No file>: line 1
2020-08-20T12:51:40.1433593Z ##[debug]at <ScriptBlock>, <No file>: line 22
2020-08-20T12:51:40.1445293Z ##[debug]at <ScriptBlock>, <No file>: line 18
2020-08-20T12:51:40.1456696Z ##[debug]at <ScriptBlock>, <No file>: line 1
2020-08-20T12:51:40.1475711Z ##[debug]Exception:
2020-08-20T12:51:40.1517637Z ##[debug]System.Management.Automation.RuntimeException: Upload to container: 'test' in storage account: 'updates' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-08-20T12:51:40.1704018Z ##[error]Upload to container: 'test' in storage account: 'updates' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-08-20T12:51:40.1706605Z ##[debug]Processed: ##vso[task.logissue type=error]Upload to container: 'test' in storage account: 'updates' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-08-20T12:51:40.1726755Z ##[debug]Processed: ##vso[task.complete result=Failed]
2020-08-20T12:51:40.2315295Z ##[section]Finishing: AzureBlob File Copy
Azure file copy to blob failing
Since you tested the task version to 2 and 3 and got the same error, I guess there might be something wrong with authentication error from Azure instead of the task itsself.
We need to make sure the Service Principal we use in this task have access to the Storage Account. You could try to navigate to Storage accounts
-> Access control page and Add a role assignment(Storage Blob Data Contributor/owner role) to my Service Principal/Managed Identity:
Besides, check also your Service Connection in your Azure DevOps Project Settings for identifying that you are using the correct Resource Group.
You could check this thread for some more details.

TFS: Exception calling "SetRight" with "2" argument(s): "Could not obtain the user information."

I have a release definition in TFS with 2 tasks. One of them passes perfectly, while the other throws an exception, though both have very smimlar configuration.
The successful task configuration:
CopyFile Task
Source: $(System.DefaultWorkingDirectory)/XXX Build Definition/drop
Machines $(WebServers)
Admin Login: server\user
Password: Password
Protocol: HTTP
Destination Folder: C:\DestFolder
Variables: WebServers: MyServer
The failing task configuration:
RunPowerShellOnRemoteComputer Task
Machines $(WebServers)
Admin Login: server\user
Password: Password
Protocol: HTTP
PowerShell Script: C:\...\script.ps1
Variables: WebServers: MyServer
When running the second task I receive this exception:
Exception calling "SetRight" with "2" argument(s): "Could not obtain
the user information." CategoryInfo :NotSpecified: (:) [],
MethodInvocationException FullyQualifiedErrorId :Exception --->
System.Management.Automation.RemoteException: Exception calling
"SetRight" with "2" argument(s): "Could not obtain the user
information."
--- End of inner exception stack trace ---
at Microsoft.VisualStudio.Services.DevTestLabs.Deployment.Deployment.PowershellExecutor.Invoke(String
errorContextMessage, Boolean writeResultToLog, Boolean isCancellable)
at Microsoft.VisualStudio.Services.DevTestLabs.Deployment.Deployment.RemoteDeploymentHelper.InstallServiceInternal(String
serviceSourcePath, String serviceName, String destinationFileName)
at Microsoft.VisualStudio.Services.DevTestLabs.Deployment.Deployment.RemoteDeploymentHelper.InstallService(String
serviceSourcePath, String serviceName, String destinationFileName)
at Microsoft.VisualStudio.Services.DevTestLabs.Deployment.Deployment.DeploymentClient.d__24.MoveNext()
---> (Inner Exception #0) System.Management.Automation.RemoteException: Exception calling
"SetRight" with "2" argument(s): "Could not obtain the user
information."<---
Deployment status for machine 'bgsovswops.eu.hpecorp.net:5985' : 'Failed' ##[error], MethodInvocationException\r\n
FullyQualifiedErrorId :Exception\r\n"}};]
##[error]System.Management.Automation.RuntimeException: Failed to install
'VisualStudioRemoteDeployer3e199746-ae71-4ba0-a2ef-f5ce8f25a631' from
service executable path VisualStudioRemoteDeployer.exe . Consult the
logs below: Exception calling "SetRight" with "2" argument(s): "Could
not obtain the user information." CategoryInfo :NotSpecified: (:)
[], MethodInvocationException FullyQualifiedErrorId :Exception
---> System.Management.Automation.RuntimeException: Failed to install 'VisualStudioRemoteDeployer3e199746-ae71-4ba0-a2ef-f5ce8f25a631' from
service executable path VisualStudioRemoteDeployer.exe . Consult the
logs below: Exception calling "SetRight" with "2" argument(s): "Could
not obtain the user information." CategoryInfo :NotSpecified: (:)
[], MethodInvocationException FullyQualifiedErrorId :Exception
--- End of inner exception stack trace ---
at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable
input)
at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace
rs, Boolean performSyncInvoke)
at System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Runspace
rsToUse, Boolean isSync)
at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection1
input, PSDataCollection1 output, PSInvocationSettings settings)
at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection1
input, PSDataCollection1 output, PSInvocationSettings settings)
at Microsoft.TeamFoundation.DistributedTask.Handlers.LegacyVSTSPowerShellHost.VSTSPowerShellHost.Main(String[]
args)
When I go to the remote machine, I succesfully execute the 'Invoke-Expression' on the C:...\script.ps1.
Also, I successfully connect from the local machines' powerscript console to the remote machine.
Any suggestions appreciated.
Please include the domain in the username variable. If it’s not domain environment, it should be the machine name.
So, just try to update the UPN user name format to DOMAIN\Username or MACHINENAME\Username format.
Check NetLogon Windows Service
The target server that I was deploying to reported this error. I already had the username for the PowerShell on Target Machines task set to use full DOMAIN\Username, per Andy's answer.
It turned out that the machine's NetLogon service had not started properly when the machine booted, due to it running on an abnormally busy VSphere cluster.
A reboot of the machine ensured that the NetLogon Windows Service, and its dependent services were all started, following which a redeploy succeeded.

Docker for windows failing to start

I've just upgraded to the Windows 10 Pro creators update | 1703 | OS Build: 15063.138.
When docker tries to start I get the following error:
Unable to create: You cannot call a method on a null-valued expression.
at <ScriptBlock>, <No file>: line 137
at Get-NetAdapter<End>, <No file>: line 181
at New-Switch, <No file>: line 137
at <ScriptBlock>, <No file>: line 381
at Docker.Backend.HyperV.RunScript(String action, Dictionary`2 parameters)
at Docker.Backend.ContainerEngine.Linux.Start(Settings settings)
at Docker.Core.Pipe.NamedPipeServer.<>c__DisplayClass8_0.<Register>b__0(Object[] parameters)
at Docker.Core.Pipe.NamedPipeServer.RunAction(String action, Object[] parameters)
Now I've tried all the usual troubleshooting methods to try to fix it but have had no luck i.e. uninstall docker reninstall, turn off hyper-V then turn it back on again.
I've even tried running the following commands using Powershell in C:\Program Files\Docker\Docker\resources
.\MobyLinux.ps1 -Destroy
.\MobyLinux.ps1 -Create
The create task throws the following error:
? : You cannot call a method on a null-valued expression.
At C:\Program Files\Docker\Docker\resources\MobyLinux.ps1:137 char:37
+ ... Get-NetAdapter | ? { $_.Name.StartsWith("vEthernet ($SwitchName)") }
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [Where-Object], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull,Microsoft.PowerShell.Commands.WhereObjectCommand
Any ideas how to fix this?
For anyone experiencing the same what i did to fix it was.
Uninstall Docker and Remove Hyper-V from the control panel. Go into
device manager -> network adapters.
Right click and uninstall any that are not your actual physical network card
Reboot and Turn Hyper-V back on and reinstall Docker (Or just install Docker as it turns on Hyper-V for you).

WindowsAzure Assembly not found while downloading Hive output on HDInsight

The following script runs a hive job from my local machine running Windows 8.1 and copies the output to a local file. It has been working fine up to now, but after upgrading to Windows 10 it is erroring out when I call Get-AzureHDInsightJobOutput. I have tried uninstalling Windows Azure Powershell, and reinstalling, but that did not help. Not sure how to proceed now, anyone have any ideas?
# Retrieve a single tagged stream from project hive database into a single file
#
#
param(
[string]$table = 'sensor_part',
[string]$bld = 'B24',
[string]$tag = 'B24/AHU/LR/_1/OAT'
)
$clusterName = 'Crunch88'
$HiveQuery = "select * from ${table} where bld='${bld}' and tag='${tag}'"
$jobDefinition = New-AzureHDInsightHiveJobDefinition -Query $HiveQuery -StatusFolder '/tagtransfer'
$job = Start-AzureHDInsightJob -Cluster $clusterName -JobDefinition $jobDefinition
Write-Host "Wait for the Hive job to complete ..." -ForegroundColor Green
Wait-AzureHDInsightJob -Job $job -WaitTimeoutInSeconds 72000
Get-AzureHDInsightJobOutput -StandardError -JobId $job.JobId -Cluster $clusterName
Write-Host "Copy the standard output to a local file" -ForegroundColor Green
Get-AzureHDInsightJobOutput -Cluster $clusterName -JobId $job.JobId -DownloadTaskLogs -TaskLogsDirectory "tasklogs"
# AzCopy /Y /Source:${blobcontainer} /Dest:${tsvpath} /SourceKey:${blobkey} /Pattern:${dirout}/stdout
Here is the results of a run with the error message:
PS C:\Windows\System32\WindowsPowerShell\v1.0> F:\88acres\3-hive\hiveretrieval\error.ps1
WARNING: When submitting a query use the -RunAsFile switch to prevent errors with query lengths or special characters
Wait for the Hive job to complete ...
Cluster : Crunch88
ExitCode : 0
Name : Hive: select * from sensor
PercentComplete : map 100% reduce 0%
Query : select * from sensor_part where bld='B24' and tag='B24/AHU/LR/_1/OAT'
State : Completed
StatusDirectory : /tagtransfer
SubmissionTime : 30-Aug-15 12:26:18
JobId : job_1438448805059_0015
Get-AzureHDInsightJobOutput : Could not load file or assembly 'Microsoft.WindowsAzure.Storage, Version=3.0.3.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
At F:\88acres\3-hive\hiveretrieval\error.ps1:18 char:1
+ Get-AzureHDInsightJobOutput -StandardError -JobId $job.JobId -Cluster ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-AzureHDInsightJobOutput], FileNotFoundException
+ FullyQualifiedErrorId : System.IO.FileNotFoundException,Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.PSCmdlets.GetAzu
reHDInsightJobOutputCmdlet
Copy the standard output to a local file
Get-AzureHDInsightJobOutput : Could not load file or assembly 'Microsoft.WindowsAzure.Storage, Version=3.0.3.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
At F:\88acres\3-hive\hiveretrieval\error.ps1:21 char:1
+ Get-AzureHDInsightJobOutput -Cluster $clusterName -JobId $job.JobId - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-AzureHDInsightJobOutput], FileNotFoundException
+ FullyQualifiedErrorId : System.IO.FileNotFoundException,Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.PSCmdlets.GetAzu
reHDInsightJobOutputCmdlet
Update:
I have enabled fusion logging. The results indicate that it is binding to 5.0.0.0 instead of 3.0.3.0. Not sure how to fix this.
Failed log entry for 3.0.3.0 follows:
*** Assembly Binder Log Entry (04-Sep-15 # 08:54:11) ***
The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.
Assembly manager loaded from: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable C:\Windows\System32\WindowsPowerShell\v1.0\powershell_ise.exe
--- A detailed error log follows.
=== Pre-bind state information ===
LOG: DisplayName = Microsoft.WindowsAzure.Storage, Version=3.0.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
(Fully-specified)
LOG: Appbase = file:///C:/Windows/System32/WindowsPowerShell/v1.0/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = powershell_ise.exe
Calling assembly : Microsoft.Hadoop.Client, Version=1.5.9.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35.
===
LOG: This bind starts in LoadFrom load context.
WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with Assembly.Load().
LOG: Using application configuration file: C:\Windows\System32\WindowsPowerShell\v1.0\powershell_ise.exe.Config
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Post-policy reference: Microsoft.WindowsAzure.Storage, Version=3.0.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage.DLL.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage/Microsoft.WindowsAzure.Storage.DLL.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage.EXE.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage/Microsoft.WindowsAzure.Storage.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft SDKs/Azure/PowerShell/ServiceManagement/Azure/HDInsight/Microsoft.WindowsAzure.Storage.DLL.
LOG: Assembly download was successful. Attempting setup of file: C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\HDInsight\Microsoft.WindowsAzure.Storage.dll
LOG: Entering run-from-source setup phase.
LOG: Assembly Name is: Microsoft.WindowsAzure.Storage, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
WRN: Comparing the assembly name resulted in the mismatch: Major Version
ERR: The assembly reference did not match the assembly definition found.
ERR: Run-from-source setup phase failed with hr = 0x80131040.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft SDKs/Azure/PowerShell/ServiceManagement/Azure/HDInsight/Microsoft.WindowsAzure.Storage/Microsoft.WindowsAzure.Storage.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft SDKs/Azure/PowerShell/ServiceManagement/Azure/HDInsight/Microsoft.WindowsAzure.Storage.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft SDKs/Azure/PowerShell/ServiceManagement/Azure/HDInsight/Microsoft.WindowsAzure.Storage/Microsoft.WindowsAzure.Storage.EXE.
LOG: All probing URLs attempted and failed.
***
Assembly Binder Log Entry (04-Sep-15 # 08:54:11) ***
And here is the successful version for 5.0.0.0:
*** Assembly Binder Log Entry (04-Sep-15 # 08:54:10) ***
The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.
Assembly manager loaded from: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable C:\Windows\System32\WindowsPowerShell\v1.0\powershell_ise.exe
--- A detailed error log follows.
=== Pre-bind state information ===
LOG: DisplayName = Microsoft.WindowsAzure.Storage, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
(Fully-specified)
LOG: Appbase = file:///C:/Windows/System32/WindowsPowerShell/v1.0/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = powershell_ise.exe
Calling assembly : Microsoft.WindowsAzure.Commands.Sync, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35.
===
LOG: This bind starts in LoadFrom load context.
WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with Assembly.Load().
LOG: Using application configuration file: C:\Windows\System32\WindowsPowerShell\v1.0\powershell_ise.exe.Config
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Post-policy reference: Microsoft.WindowsAzure.Storage, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage.DLL.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage/Microsoft.WindowsAzure.Storage.DLL.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage.EXE.
LOG: Attempting download of new URL file:///C:/Windows/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Storage/Microsoft.WindowsAzure.Storage.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft SDKs/Azure/PowerShell/ServiceManagement/Azure/Compute/Microsoft.WindowsAzure.Storage.DLL.
LOG: Assembly download was successful. Attempting setup of file: C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Compute\Microsoft.WindowsAzure.Storage.dll
LOG: Entering run-from-source setup phase.
LOG: Assembly Name is: Microsoft.WindowsAzure.Storage, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
LOG: Where-ref bind Codebase does not match what is found in default context. Keep the result in LoadFrom context.
LOG: Binding succeeds. Returns assembly from C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Compute\Microsoft.WindowsAzure.Storage.dll.
LOG: Assembly is loaded in LoadFrom load context.
Update: Added the required assembly using a Nuget package. Still didn't help. Think I somehow need to tell Nuget to make it available for the PowerShellISE, but don't see a parameter for that... This must be close to a solution.
PS C:\WINDOWS\system32> nuget install WindowsAzure.Storage -version 3.0.3
Attempting to resolve dependency 'Microsoft.Data.OData (ò 5.6.0)'.
Attempting to resolve dependency 'System.Spatial (= 5.6.0)'.
Attempting to resolve dependency 'Microsoft.Data.Edm (= 5.6.0)'.
Attempting to resolve dependency 'Newtonsoft.Json (ò 5.0.6)'.
Attempting to resolve dependency 'Microsoft.Data.Services.Client (ò 5.6.0)'.
Attempting to resolve dependency 'Microsoft.WindowsAzure.ConfigurationManager (ò 1.8.0.0)'.
Installing 'System.Spatial 5.6.0'.
Successfully installed 'System.Spatial 5.6.0'.
Installing 'Microsoft.Data.Edm 5.6.0'.
Successfully installed 'Microsoft.Data.Edm 5.6.0'.
Installing 'Microsoft.Data.OData 5.6.0'.
Successfully installed 'Microsoft.Data.OData 5.6.0'.
Installing 'Newtonsoft.Json 5.0.6'.
Successfully installed 'Newtonsoft.Json 5.0.6'.
Installing 'Microsoft.Data.Services.Client 5.6.0'.
Successfully installed 'Microsoft.Data.Services.Client 5.6.0'.
Installing 'Microsoft.WindowsAzure.ConfigurationManager 1.8.0.0'.
Successfully installed 'Microsoft.WindowsAzure.ConfigurationManager 1.8.0.0'.
Installing 'WindowsAzure.Storage 3.0.3.0'.
Successfully installed 'WindowsAzure.Storage 3.0.3.0'.
PS C:\WINDOWS\system32> F:\88acres\3-hive\hiveretrieval\error.ps1
WARNING: When submitting a query use the -RunAsFile switch to prevent errors with query lengths or special characters
Wait for the Hive job to complete ...
Cluster : Crunch88
ExitCode : 0
Name : Hive: select * from sensor
PercentComplete : map 100% reduce 0%
Query : select * from sensor_part where bld='B24' and tag='B24/AHU/LR/_1/OAT'
State : Completed
StatusDirectory : /tagtransfer
SubmissionTime : 04-Sep-15 08:03:50
JobId : job_1440975278889_0021
Get-AzureHDInsightJobOutput : Could not load file or assembly 'Microsoft.WindowsAzure.Storage, Version=3.0.3.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
At F:\88acres\3-hive\hiveretrieval\error.ps1:18 char:1
+ Get-AzureHDInsightJobOutput -StandardError -JobId $job.JobId -Cluster ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-AzureHDInsightJobOutput], FileNotFoundException
+ FullyQualifiedErrorId : System.IO.FileNotFoundException,Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.PSCmdlets.GetAzu
reHDInsightJobOutputCmdlet
I didn't find a good solution for this issue. I ran into this same issue after I installed to Visual Studio 2015, so if someone has a good solution I'd love to hear it. At this point in time... I just took the Microsoft.WindowsAzure.Storage.dll version 3.0.3.0 and copied it to my \Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\HDInsight folder (making a backup of the newer version first). It works fine now, but this is only a temporary work around.

Running tests with tcm.exe results in Field not found: 'Microsoft.TeamFoundation.TestManagement.Common.WitCategoryRefName.SharedDataSet'

I'm trying to run an automated test case in Microsoft Test Manager from the command line with the following command:
TCM.exe run /create /title:"Nightly Run" /planid:5554 /suiteid:6582 /configid:97 /collection:XXX /teamproject:XXX /include /builddir:'C:\Source\'
This results in the following error:
.\TCM.exe : Field not found: 'Microsoft.TeamFoundation.TestManagement.Common.WitCategoryRefName.SharedDataSet'.
At C:\Users\XXXX\Desktop\RunTest.ps1:2 char:1
+ .\TCM.exe run /create /title:"Nightly Run" /planid:5554 /suiteid:6582 /configid: ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Field not found...SharedDataSet'.:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
However, this error happens on a Windows Server 2012R2 machine with Visual Studio 2013.4 installed but not on my local dev machine (Windows 10 with VS2013.4).
I know that SharedDataSets are introduced in VS2013.4 but I don't know where this error is coming from since the command doesn't give this error on my local pc.
I've found the solution. Running the following commands on the Windows Server fixed the error:
ngen uninstall Microsoft.TeamFoundation.TestManagement.Client
ngen uninstall Microsoft.TeamFoundation.TestManagement.Common