If you want to configure your VS "Load Tests" to write the results to a database server, you use the following instructions.
If you want to run your "Load Tests" through powershell on a separate machine(think TFS 2018 release step), you use the following instructions.
I would like to do both, on multiple machines, in a automated manner, but there's not a great deal of documentation on this, I can run my tests like this:
.\mstest /testcontainer:"C:\XXX\ABC.loadtest"
But the results are kicked out to a "TRX" file rather than being placed into a database(there is some discussion on this). How do I put the results into a external database like when I run it locally(per instructions above)?
Note: #AdrianHHH points out that the "TRX" file is only a summary and that most of the info is stored locally(MDF/LDF file) in the user folder of current user running the load tests.
Update 1
Hmm I wonder where this is persisted:
(Curious, also click on the "?" icon in the "Manage Test Controller" box, nothing...)
It's not in the saved XML:
<RunConfigurations>
<RunConfiguration Name="Run Settings1" Description="" ResultsStoreType="Database" TimingDetailsStorage="AllIndividualDetails" SaveTestLogsOnError="true" SaveTestLogsFrequency="0" MaxErrorDetails="200" MaxErrorsPerType="1000" MaxThresholdViolations="1000" MaxRequestUrlsReported="1000" UseTestIterations="false" RunDuration="10" WarmupTime="0" CoolDownTime="0" TestIterations="100" WebTestConnectionModel="ConnectionPerUser" WebTestConnectionPoolSize="50" SampleRate="5" ValidationLevel="High" SqlTracingConnectString="" SqlTracingConnectStringDisplayValue="" SqlTracingDirectory="" SqlTracingEnabled="false" SqlTracingFileCount="2" SqlTracingRolloverEnabled="true" SqlTracingMinimumDuration="500" RunUnitTestsInAppDomain="true" CoreCount="0" ResourcesRetentionTimeInMinutes="0" AgentDiagnosticsLevel="Warning">
<CounterSetMappings>
<CounterSetMapping ComputerName="[CONTROLLER MACHINE]">
<CounterSetReferences>
<CounterSetReference CounterSetName="LoadTest" />
<CounterSetReference CounterSetName="Controller" />
</CounterSetReferences>
</CounterSetMapping>
<CounterSetMapping ComputerName="[AGENT MACHINES]">
<CounterSetReferences>
<CounterSetReference CounterSetName="Agent" />
</CounterSetReferences>
</CounterSetMapping>
</CounterSetMappings>
<LoadGeneratorLocations>
<GeoLocation Location="Default" Percentage="100" />
</LoadGeneratorLocations>
</RunConfiguration>
</RunConfigurations>
They're not persisted in my default "testsettings" file either:
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local" id="02cad612-043b-447d-993e-a9b9b0547c9d"
xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>These are default test settings for a local test run.</Description>
<Deployment enabled="false" />
<Execution hostProcessPlatform="MSIL">
<TestTypeSpecific />
<AgentRule name="Execution Agents">
</AgentRule>
</Execution>
<Properties>
<Property name="TestSettingsUIType" value="UnitTest" />
</Properties>
</TestSettings>
So I need to find where ever this configuration information is being persisted, then maybe I can find a way to feed it to MSTest. Does anyone else understand how this works?
Update 2
My TRX file does contains a "connection string" but I don't think it's to my database, my database is empty, on running via powershell it completes, but all I see is the "TRX" file.
Update 3
This one is tricky, I keep trying various ways to determine where this "Manage Test Configuration" data/credentials is being stored. One of the ways I did this was to use Microsoft's Process Monitor. You can actually see where it initially is being populated from:
It's from a Application Hive, of course that's begs the question where did the "Application Hive" get populated from, that's where things get a bit murky, there's allot of different calls to many files. A common trend is that the "Temp\Local" folder is often referenced.
I deleted the entire "Temp" folder for my user account(in the process losing all my VS configuration) and upon reopening my solution it appears as though this had an effect. When I pull up my "LoadTest" file, the "Load test results store" line is now empty. In fact the entire "Manage Test Controller" window has been restored back to default(empty).
I know believe that the configuration for this "Manage Test Controller" window is persisted in the temp folder. However, I've yet to locate where it is and/or how to change/automatically populate that information with a powershell script.
Finally figured this out. Basically I used several tools to check what files were being modified when I changed the connection string, the results made it obvious:
privateregistry.bin
Once I found this it was pretty obvious that VS was maintaining it's own little registry hive. It's clearly stated in this post, so I opened it in the way described in the article and found the connection string:
This indicated that:
"The SQL Connection String is NOT stored in the loadtest files. The
setting seems to be PC specific so I had to change it on the build
server - in one loadtest file (address.loadtest) as shown, then all
the other loadtests adopt the same connection string."
So that's basically what I did, I logged into each build server and configured them so that they write all there results to my database rather than locally.
Load tests are clearly not designed to make this process easy, I don't think many people have attempted to do what I've done. All the articles just tell you to use their cloud service. I'm pretty sure that only covers web tests. If your using load testing to test unit tests you pretty much out of luck(without this work around). I really hope this gets official support in the future, it would be really nice to both run/view all types of load tests from TFS. For now though I'm going to have to keep using this work around.
I have a scenario where I need to read an environment variable from the machine where my Service Fabric application is deployed.
More specifically in my cloud.xml (environment file) I want StorageConnectionString to use the value from one of the environment variable that is set on the machine by some other external tool.
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/AppFabricName.ServiceFabric" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="StorageConnectionString" Value="%ENVVARIABLE%" />
</Parameters>
</Application>
Is the above valid ? Did not work when I tried even though running SET on cmd prompt did show that variable exists.
It's not straight forward, but here's a way:
Specify a setup entry point (a new executable or script) to read the environment variable and write it to a file in the application's work directory.
When the main entry point comes up, read the file from the work directory.
If you are passing the data via configuration files(or parameters) at deployment and accessing it via code using Context.CodePackageActivationContext.GetConfigurationPackageObject("Config") , the right way should be using <Parameter Name="StorageConnectionString" Value="[ENVVARIABLE]" /> (note that it is encolsed by [ ]) and have a configuration file with a variable called ENVVARIABLE and set the overrides to your service. You should not need to set environment variables, environment variables is other thing.
Please take a look at this SO question to check if will solve the problem.
If your plan is getting an environment variable already set in the machine, unfortunately SF does not support Environment Variable Replacement using tokens like %ENVVARIABLE%.
even though running SET on cmd prompt did show that variable exists.
The other process that changes the environment variable, if it just use the default set, it will set their own process environment variables, to be available to other processes, it has to update environment variable on Machine or User scope(assuming they run on same user), this is why you can't see from command running SET variablename
...
Assuming the variable is set correctly and it is a machine scope variable...
When the Environment Variable is set before the process start, I would suggest you setup an EntryPoint Script to set it at startup, like you would do to a guest executable.
See a NodeJs example here
You could maybe also do write to file and read from your app as Loek suggested, but I think it would be too complicate for too litle.
In case the environment variable is set after the process started, or if it changes later, you should get it from within your application directly instead, you could just use:
System.Environment.GetEnvironmentVariable(variableName,EnvironmentVariableTarget.Machine);
And you pass the variable name via config.
In both cases your user\process must have permission to read Environment Variable from Machine Scope
I have a little bit of experience using Capistrano but only on projects that are already using it and I am currently trying to set it up on a new project. I have created my config/deploy.rb file, added the appropriate configuration and am now trying to run "cap deploy:setup" to set the correct capistrano structure up on my remote server however now when I try to run this or any other Capistrano commands I get:
/Library/Ruby/Gems/2.0.0/gems/capistrano-2.15.5/lib/capistrano/configuration/loading.rb:93:in `instance_eval': ./config/deploy.rb:8: no . floating literal anymore; put 0 before dot (SyntaxError)
role :web, xxx.x.xx.xx
It looks like an issue with the format of the IP that I have provided for my app but no variations seem to work.
Am I missing something obvious here?
Has anyone else come across this issue?
Thanks
James
I found a resolution for this in the end through trial and error. It turns out that I needed to wrap the role :web and role :app values in quotes:
"xx.xx.xx.xx"
Then it worked correctly.
I'm using the github oauth plugin for our logins but for all of our users in the Organisation I get an error:
Access Denied
<user> is missing the Overall/Read permission
I have tried everything I can possibly think of to try to make this work and I'm probably going to fallback to making everyone an admin user, which i would prefer not to do.
Any advise would be appreciated.
This is how I resolved the authentication problem:
Edit config.xml file, e.g.
sudo vi /var/lib/jenkins/config.xml
Change useSecurity element's value to false, e.g.
<useSecurity>false</useSecurity>
Remove authorizationStrategy block
Restart Jenkins: /etc/init.d/jenkins restart.
Access Jenkins through URL as usual and reconfigure security again.
I had the same problem with "... is missing the Overall/Read permission" on Jenkins (1.651.2) with activated Credentials Plugin.
But it was my own failure: I only configured the user on project side (by credential plugin) but missed to configure the global security.
So I fixed it by selecting:
Jenkins -> Manage Jenkins -> Configure Global Security
And did setup missing global settings (or project matrix based one)
Have you followed this step, from the plugin page?
Control user authorization (i.e. who is allowed to see the jobs and build them) using the Github Commiter Authorization Strategy
Also, make sure you actually allow authenticated users to access Jenkins
Under Jenkins global configuration, under Authorization, add user/group called authenticated
Give that group Overall Read permission
The group should show up with a "group" icon (two users), as opposed to single user icon.
reset from <useSecurity>true</useSecurity> to <useSecurity>false</useSecurity> in config.xml and set the permission again.
Edit file /var/lib/jenkins/config.xml and add the following lines :
<authorizationStrategy class="hudson.security.ProjectMatrixAuthorizationStrategy">
<permission>hudson.model.Hudson.Read:john.smith</permission>
</authorizationStrategy>
Restart Jenkins
What I did when I got this error is to edit config.xml as mentionned by other users and correctly re-add my username in LOWERCASE in "configureSecurity" Jenkins' page. I was using "KrustyHack" when adding permissions but it didn't work. I had to add "krustyhack" instead, and it worked.
I hope it helps.
I had the same problem here, but it affected only some users, not all of them. Anyway, you should check public organization membership : documentation of the plugin states that "You have to be a public member of the organization for the authorization to work correctly." (https://wiki.jenkins-ci.org/display/JENKINS/Github+OAuth+Plugin).
Follow instructions from GitHub (https://help.github.com/articles/publicizing-or-hiding-organization-membership/) in order to make organization membership public, and this might fix your issue.
Also check the case user names in the authorizationStrategy element. I made my new user's name lower case and restarted the service and it the error went away.
Fix it by these 2 shell commands on the server (sudo permission is required):
sudo ex +g/useSecurity/d +g/authorizationStrategy/d -scwq /var/lib/jenkins/config.xml
sudo /etc/init.d/jenkins restart
This will remove useSecurity and authorizationStrategy lines from your config file.
See also: Disable security at Jenkins website
We hit this same error when a github organization administrator changed the organization's settings for "Third-party access" to "restrict third-party application access". Reverting to the previous settings within the github organization resolved the problem.
See github oauth-app-access-restrictions for details on how to configure that properly.
The assignment of roles to users is stored in config.xml file. Add the ID of the user directly to the role and then restart Jenkins.
In my case, I have a role named editor and a bunch of users assigned to the role.
<role name="editor" pattern=".*">
<permissions>
<...>
<permission>hudson.model.Item.Create</permission>
<permission>hudson.model.Item.Workspace</permission>
<permission>...</permission>
</permissions>
<assignedSIDs>
<sid>bob</sid>
<sid>alice</sid>
<sid>**newuser**</sid>
</assignedSIDs>
</role>
The matrix security is not terribly clear. I am a member of a specific group in our org that has admin privileges however I am also an authenticated user. I would think that the one group super-cedes the other however I have to have both in order to actually log into the system and be admin. It's screwed up IMO.
go to your-jenkins-host:port/role-strategy/assign-roles and configure roles for the user
I had the same problem before,
your OAuth application need your organization owner's approve,
then the OAuth Plugin can access the private data in it
I am using Crowd 2 plugin and I have the same problem.
I fixed it by downgrading OWASP Markup Formatter Plugin from varsion 1.2 to version 1.1 and then changing Markup Formatter in Configure Global Security value to Raw HTML, before it was Plain text.
I had exactly the same problem and adding the plugin Role Strategy Plugin fixed the problem.
All I had to do was install the plugin, create two groups - admin / developer and then add users to the groups.
A much much better solution than recreating the whole permissions matrix :)
I had a similar problem I was not able access Jenkin account and the system was locked.
I had only an error message. "Access Denied "
When I tried to reinstall Jenkins then it prompted to Repair option.
By clicking Repair option it fixed the problem.
Go to $JENKINS_HOME (linux, jenkins in windows), and find config.xml file.
Open this file in the editor. (take backup of .jenkins home)
Look for the <useSecurity>true</useSecurity> element in this file.
Replace "true" with "false"
Remove the elements authorizationStrategy and securityRealm
Start Jenkins
I found it in
C:\ProgramData\Jenkins\.jenkins
Jenkins Version: 2.319.2
Instead of removing all security (the top answer), Add admin access as root, to the user you want to give admin to. We had the same issue where all admins were no longer with the company. This is how I resolved the authentication problem: I logged into:
jenkins#<jenkins server>:/var/lib/jenkins/
Edit config.xml file, and add a config for
<permission>hudson.model.Hudson.Administer:<username></permission>
Then restart Jenkins:
root#<jenkins server>:/$ /etc/init.d/jenkins restart
Just use Jenkins > Configure Global Security bottom page matrix to provide permissions to the user (start w/ read)
I edited the /var/lib/jenkins/config.xml file and replaced the
<authorizationStrategy>...</authorizationStrategy>
with
<authorizationStrategy class="hudson.security.FullControlOnceLoggedInAuthorizationStrategy">
<denyAnonymousReadAccess>true</denyAnonymousReadAccess>
</authorizationStrategy>
It is the default settings after installation. Then restart the jenkins service.
In JBoss 7.1.1 in standalone mode all JNDI bindings are configured in standalone.xml file in jboss:domain:naming:1.1 subsystem. According to documentation standalone.xml cannot be modified when server is running. I've tried to use JBoss CLI but I don't know how to write/modify resource.
How to change value in JNDI without restarting jboss?
Should help you: https://docs.jboss.org/author/display/AS71/JNDI+Reference
Topic - Binding entries to JNDI:
An example standalone.xml might look like:
<subsystem xmlns="urn:jboss:domain:naming:1.1" >
<bindings>
<simple name="java:global/a" value="100" type="int" />
<object-factory name="java:global/b" module="com.acme" class="org.acme.MyObjectFactory" />
<lookup name="java:global/c" lookup="java:global/b" />
</bindings>
</subsystem>
To add these entries via the CLI:
/subsystem=naming/binding=java\:global\/mybinding:add(binding-type=simple, type=long, value=1000)
To see all all options that are taken by the add command (this can
actually be used to get the description of any CLI command):
/subsystem=naming/binding=*:read-operation-description(name=add)
Have not tried, but i hope this helps!
UPDATE - with tested examples:
Add JDNI name binding java:global/a:
/subsystem=naming/binding=java\:global\/a:add(value=10,binding-type=simple,type=java.lang.Integer)
Read existing JDNI name binding java:global/a:
/subsystem=naming/binding=java\:global\/a:read-resource(include-defaults=true)
Modify JDNI name binding value java:global/a:
/subsystem=naming/binding=java\:global\/a:write-attribute(name=value, value=20)
Remove JDNI name binding java:global/a:
/subsystem=naming/binding=java\:global\/a:remove()
Executing command directly from shell:
./jboss-cli.sh --connect --command="/subsystem=naming/binding=java\:global\/a:read-resource(include-defaults=true)"
The question has a lot of views so I'll answer to it. Inspired by #mik response I've figured out that to change value of some JNDI key e.g. java:jboss/api/key to newApiKey run JBoss CLI and execute:
connect
/subsystem=naming/binding=java\:jboss\/api\/key/:write-attribute(name=value,value=newApiKey)
The change will be immediately visible on server and also stored (updated) in standalone.xmlso it won't get lost after server restart.
I was looking exactly for how to add or modify a JNDI binding at runtime, but I needed to to do this in a Wildfly 9 domain (cluster) configuration (not standalone), which is pretty much the same configuration as JBoss 7. However, I couldn't figure out a way to effectively apply changes without restarting all servers.
To start with, enter the JBoss command line interface and connect to your server domain controller:
./jboss-cli.sh
connect
First, you need to find which profile is active on the server group, so as, on the server root /, enter the following commands:
cd server-group=
ls
Afterwards, you should enter the only server group shown in the listing command (ls) by typing cd {{your_server_group_name}}, then type ls again and look for an entry named profile to check which one is active. Let's consider full-ha as an active profile for our example.
Next, go back to the root configuration folder / by typing cd .. and enter the following commands to navigate and view all JNDI bindings available with their current values:
cd profile=full-ha/subsystem=naming/binding=
:read-resource(recursive=true)
By doing this, you'll be able to see all available JNDI bindings and their attributes, if you want to list only binding names, type ls instead of the last command.
In order to modify a binding, type cd and the name of the binding listed in the previous command. Let's suppose you want to change the value of a binding named java:/webservice.url, then you should enter
cd java\:\/webservice.url
Notice that is necessary to quote some characters in your binding name such as : (colon) and / (slash) with a backslash (\).
To modify an attribute within this binding you should use the :write-attribute command. In this example, let's suppose you want to modify (or add) an attribute named "value" with its content as "this is a value":
:write-attribute(name=value,value="this is a value")
So as to apply this change, you'll need to restart all servers in the cluster by typing the following command:
/server-group={{server-group-name}}:restart-servers
If you want to know more commands to add or remove JNDI bindings check this jboss-cli snippets page
This configuration has been tested successfully in Wildfly 9.0.1