Submit U-SQL Script locally using Powershell - powershell

If I want to submit a U-SQL Script to ADLA account, I can do so, using the below powershell cmdlet.
`
Submit-AzureRmDataLakeAnalyticsJob -Account $adlaAccountName -ScriptPath $USQLFile -Name $folder -DegreeOfParallelism $degreeOfParallelism -Verbose`
But, I am not able to submit the script against local ADLA account. I am getting error:
Submit-AzureRmDataLakeAnalyticsJob : Run Login-AzureRmAccount to login.
At line:1 char:1
+ Submit-AzureRmDataLakeAnalyticsJob -Account "local" -ScriptPath GetIn ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [Submit-AzureRmDataLakeAnalyticsJob], PSInvalidOperationExceptio
n
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.Azure.Commands.DataLakeAnalytics.SubmitAzureDataLakeAnalyti
csJob
How to submit U-SQL script against local ADLA account ?

In addition to ADL Tools, we have released standalone SDK as well.
https://www.nuget.org/packages/Microsoft.Azure.DataLake.USQL.SDK
And document here:
https://learn.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-data-lake-tools-local-run
Thanks a lot,
Yu Dong

Related

Import pfx file with sn.exe using powershell

I am trying to install pfx file automatically every time the Azure Pipeline runs. Below is the powershell script i am trying to implement. I need it to provide the password automatically:
Set-Location "C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools"
Start-Process -FilePath "sn.exe" -ArgumentList "-d",$container
Start-Sleep 2
Start-Process -FilePath ".\sn.exe" -ArgumentList "-i","c:\cert_2013.pfx",$container -NoNewWindow
Start-Sleep 2
[void] [System.Reflection.Assembly]::LoadWithPartialName("'System.Windows.Forms")
[System.Windows.Forms.SendKeys]::SendWait("$($CerPass){ENTER}")
Start-Sleep 2
[System.Windows.Forms.SendKeys]::SendWait("{ENTER}")
When the pipeline runs the ps1 file it is erroring out at SendWait("$($CerPass){ENTER}") Step. Below is the exact error message:
Enter the password for the PKCS#12 key file:
Could not read password for PKCS#12 blob in c:\cert_2013.pfx --
Console input may not be redirected for password entry.
Exception calling "SendWait" with "1" argument(s): "Access is denied"
At line:1 char:1
+ & 'C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe' -NoLogo ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Exception calli...cess is denied":String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
At C:\PowerShellScript\CertReInstall.ps1:18 char:46
+ [System.Windows.Forms.SendKeys]::SendWait("$($CerPass){ENTER}")
+ CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : Win32Exception
[error]Non-Zero exit code: '1' for ComputerName: 'PQAFCTC1'
[error]Atleast one remote job failed. Consult logs for more details. ErrorCodes(s): 'RemoteDeployer_NonZeroExitCode'
[section]Finishing: Run PowerShell on Target Machines
I need it to be automatically send the password to the file when installing. When i run the ps1 file directly on the server it works fine, but I need this script to be part of pipeline task (azure devops pipeline).
Any help would be greatly appreciated.
Thank you!

powershell returned an error:(403) WebException using New-pnpList

I had my PS script running the other day and created a new list with a single field with no issues. Was able to view the list in the Site Contents lib.
Today, it's not working. I tried running the PS code below and then resorted to running the new-pnplist code at the command line...got the same error which is shown below.
I'm using the Global admin account. Using version 3.13.19 SharePointPNPPowerShellOnline. Executed the commands from Powershell ISE which is what I did the other day when it worked. And, I was able to connect successfully using the Connect-sposervice command.
Error:
new-pnplist -Template GenericList -Title 'TestPNP2'
new-pnplist : The remote server returned an error: (403) Forbidden.
At line:1 char:1
+ new-pnplist -Template GenericList -Title $ListName
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : WriteError: (:) [New-PnPList], WebException
+ FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.Lists.NewList
Code:
$TargetListURL="https://<my sharepoint.com>/sites/CKCDemo"
$ListName="TESTPNP2"
Connect-PnPOnline -url $TargetListURL -CurrentCredentials
New-PnPList -Template GenericList -Title $ListName
Add-PnPField -List $ListName -DisplayName "MyTEST" -InternalName "MyTEST" -Type Text -AddToDefaultView
Get-PnPList`
Make sure you have enough permission to create a list in this site. You could try to go to the site directly, check whether you could create a list through ui.
Note: A global admin will not automatically have access to individual sites unless explicitly granted.

New-AzureRmResourceGroup command not working in powershell

I have logged into my Azure account and selected the appropriate subscription.
But it always gives the same error
PS C:\WINDOWS\system32> New-AzureRmResourceGroup -Name "AzureMediaServicesSample" -Location "East US"
New-AzureRmResourceGroup : Run Login-AzureRmAccount to login.
At line:1 char:1
+ New-AzureRmResourceGroup -Name "AzureMediaServicesSample" -Location " ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [New-AzureRmResourceGroup], PSInvalidOperationException
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzur
eResourceGroupCmdlet
I have selected the subscription too using select-azuresubscription
select-azuresubscription is used for ASM mode.
For ARM mode, you need use Get-AzureRmSubscription -SubscriptionName "your sub" | Select-AzureRmSubscription to select your subscription.
More information please refer to this example.
Update:
You could use the following cmdlet to check Azure PowerShell version.
Get-Module -ListAvailable -Name Azure -Refresh
You could install the latest PowerShell from the link.
I had the same problem and I solved it by changing the environment setting from Bash to Powershell in the upper left corner of the Cloud Shell Window.

how do I create a new azure management cache?

Now that Microsoft has completely dropped the GUI support on the management portal, they are requiring me to install and setup and use windows Azure PowerShell to create a new managed cache service. As someone with not a ton of PowerShell experience. I cannot figure out how to successfully create a new azure managed cache. Does anyone have any tips?
I did the following:
executed in PowerShell:
Add-AzureAccount
connected with login to Azure account
executed in PowerShell:
New-AzureManagedCache -Name test -Location "South Central"
and the error is:
New-AzureManagedCache : Value cannot be null.
Parameter name: managementCertificate
At line:1 char:1
+ New-AzureManagedCache -Name test-Location "South Central"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [New-AzureManagedCache], ArgumentNullException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.ManagedCache.NewAzureManagedCache
What should I do to fix this issue and get a new basic cache 128mb created?
i figured it out - i needed to
get-azurepublishsettingsfile
and then
import-azurepublishsettignsfile "c:\filename.publishsettings"
and then it appears to be creating it now
With explicit sku paramater:New-AzureManagedCache -Name test -Location "South Central" -Sku Basic -Memory 128MB

Remote Powershell Access denied for certain dll's execution for Sharepoint 2013

I am attempting to automate a sharepoint 2013 deployment via remote powershell from the build server. Everything executes as expected except when having anything to do with some class in sharepoint dll's such as (Microsoft.SharePoint.Publishing, Microsoft.SharePoint.Publishing.Navigation.WebNavigationSettings)
If I run the same script locally under the same credentials it runs fine.
I have considered the below:
The user has full admin right on both machines
Disabled UAC on the remote server
Followed the required Remote Powershell steps in thig post (http://social.technet.microsoft.com/Forums/sharepoint/en-US/09b60466-5432-48c9-aedd-1af343e957de/user-cannot-be-found-when-using-invokecommand-newspsite-on-sharepoint)
I set powershell to run as admin by defualt via the registry (New-Item -Path "Registry::HKEY_CLASSES_ROOT\Microsoft.PowershellScript.1\Shell\runas\command" -Force -Name '' -Value '"c:\windows\system32\windowspowershell\v1.0\powershell.exe" -noexit "%1"')
Script Code:
#Set the radio buttons value
$settings = New-Object Microsoft.SharePoint.Publishing.Navigation.WebNavigationSettings (,$rootWeb)
$settings.GlobalNavigation.Source = [Microsoft.SharePoint.Publishing.Navigation.StandardNavigationSource]::PortalProvider
#Set the radio buttons value
$settings.CurrentNavigation.Source = [Microsoft.SharePoint.Publishing.Navigation.StandardNavigationSource]::PortalProvider
write-host "I am here.........................."
$settings.Update()
#Set the Publishing Web
$SPPubWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($rootWeb)
#Global Navigation Settings
$SPPubWeb.Navigation.InheritGlobal = $false
$SPPubWeb.Navigation.GlobalIncludePages = $false
The Remote Powershell output is as below:
I am here..........................
Exception calling "Update" with "0" argument(s): "Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))"
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : UnauthorizedAccessException
+ PSComputerName : Contoso-DEVSP
Exception setting "GlobalIncludePages": "Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))"
+ CategoryInfo : NotSpecified: (:) [], SetValueInvocationException
+ FullyQualifiedErrorId : ExceptionWhenSetting
+ PSComputerName : Contoso-DEVSP
Many thanks in advance
You need to check CredSSP authentication. Remote PowerShell execution with SharePoint fails as the second hop translates the credentials to system credentials. If the task involves querying or updating DB server, it will fail as SYSTEM account will not have access the remote PowerShell on SQL Server. You need to enable CredSSP.
Check this blog post I wrote a while ago. This is not specific to SharePoint but it should apply to your scenario as well.
http://www.ravichaganti.com/blog/?p=1230