Is it possible to replace a file from one repository to another with android repo tool? - yocto

I'm using repo tool to build a Yocto project, the repositories used are A, B, yocto ..., and I need to replace a file from A to B, the structure is something like this:
A/MyFile.sh
B/TheFile.sh
yocto/Some_dirs_and_files
So, I use the copyfile like this:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote fetch="mygitrepo" name="origin"/>
<default remote="origin"/>
<project name="yocto" revision="myrevision"/>
<project name="meta-openembedded" path="yocto/meta-openembedded" revision="myrevision"/>
<project name="B" path="yocto/B" revision="myrevision"/>
<project name="C" path="yocto/meta-swi-extras" revision="myrevision"/>
<project name="poky" path="yocto/poky" revision="myrevision"/>
<project name="A" path="yocto/custom-builds" revision="myrevision">
<copyfile src="MyFile.sh" dest="yocto/B/TheFile.sh"/>
</project>
</manifest>
The problem is that the copyfile is not replacing the file "TheFile.sh" with "MyFile.sh"
Is there a way to do it without an additional script?
Note: If I change the dest name from
dest="yocto/B/TheFile.sh
to
dest="yocto/B/AnotherFile.sh
the file is succesfully copied, but if I set the name to the file I want to replace it doesn't.

It seems repo do now allow overwrite file by <copyfile src=.. dest ...>
From repo source code project.py
class _CopyFile(object):
def __init__(self, src, dest, abssrc, absdest):
self.src = src
self.dest = dest
self.abs_src = abssrc
self.abs_dest = absdest
def _Copy(self):
src = self.abs_src
dest = self.abs_dest
# copy file if it does not exist or is out of date
if not os.path.exists(dest) or not filecmp.cmp(src, dest): ※
※line shows the condition to do a file copy.

code below your pice
if not os.path.exists(dest) or not filecmp.cmp(src, dest):
try:
# remove existing file first, since it might be read-only
if os.path.exists(dest):
platform_utils.remove(dest)
else:
dest_dir = os.path.dirname(dest)
if not platform_utils.isdir(dest_dir):
os.makedirs(dest_dir)
shutil.copy(src, dest)
# make the file read-only
mode = os.stat(dest)[stat.ST_MODE]
mode = mode & ~(stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH)
os.chmod(dest, mode)
so it can replace ...

is this still open?
I just happened to come across the same issue, wanting to overwrite a file from repository2 with a file from repository1, the <copyfile> was in place to replace myfile from repository2 with myfile from repository1. I was using a yocto distribution (with several layers, as git repositories).
But it did not work.
Copying myfile from repository1 as myfile2 to repository2 (another name) worked.
What I discovered was that running the repo init/sync commands several times I did not get the repos to be populated in the same order.
So basically my <copyfile> did what is was supposed to do when repository1 was populated, but that happened before repository2 was populated (even if they were in the right order in the manifest file). And repository2 simply brought its own myfile, overwriting the one copied by repository1.
My mega-solution was to use two <copyfile> tags: one in repository1 to copy myfile as myfile2 into repository2, and the other in repository2 to copy myfile2 as myfile.
You have to make sure though that repository1 is always populated before repository2.
This all thing is very strange, since repo does not guarantee the order in which repositoruies are populated.

Related

Update Build number in App config xml file on build pipeline

I have a build pipeline in Azure DevOps, I need to update the build number in my apconfig exe file that will be $(Build.BuildNumber).
I just tried this way:
Adding a variable name = BuildNumber value = $(Build.BuildNumber).
And in my apconfig.exe file have a key same like <add key="BuildNumber" value="1812201901" />.
Why I have tried like this way: thinking like it will update in the config file if variable name match with the key.
But it is not working. can anyone please help? I have just started in CI/CD.
Update Build number in App config xml file on build pipeline
Just like the Shayki said, using the Replace Tokens extension should be the directly way to resolve this issue.
But since you need to request to get this extension, as workaround, you could also use power shell scripts to resolve this issue, you can check below my test powershell scripts:
$currentDirectory = [IO.Path]::GetDirectoryName($MyInvocation.MyCommand.Path)
$appConfigFile = [IO.Path]::Combine($currentDirectory, 'App.config')
$appConfig = New-Object XML
$appConfig.Load($appConfigFile)
foreach($BuildNumber in $appConfig.configuration.add)
{
'name: ' + $BuildNumber.name
'BuildNumber: ' + $BuildNumber.value
$BuildNumber.value = '123456789'
}
$appConfig.Save($appConfigFile)
As result, the app.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
</startup>
<add key="BuildNumber" value="123456789" />
</configuration>
Note: Set the powershell scripts at the same folder of the app.config file.
Hope this helps.
You can use the Replace Tokens extension and in the apconfig.exe file put this:
<add key="BuildNumber" value="__BuildNumber__" />
Configure the task to search variables with __ prefix and suffix:
Now the value will be replaced with the value of the BuildNumber variable you configured (equal to Build.BuildNumber).

Azure DevOps Release Pipeline Web.Config Edit

I know that when creating a release pipeline in Azure DevOps you can have the web.config of an app updated with variables from the pipeline and that works great for all the appSettings values.
But, during the release pipeline I'd like to update a different section of the web.config, specifically the sessionState provider node. I've tried a few plugins for the release pipeline like Config Transform by Magic Chunks but the problem is it needs you to specify the path of the configuration file to edit but by the time it gets to the release pipeline the source files are in a zip archive. Somehow the normal transformations of the appSettings are able to work off the unzipped version but I can't get other transformations to happen after the file is unzipped.
I know you can make changes in the build pipeline but there are reasons we want to do it in the release pipeline.
Anyone know a way to make changes to the web.config outside of the appSettings grouping in a release pipeline for an Azure App Service?
You can use PowerShell to do the transformation within the zip file.
For example, I have this node in the web.config:
<configuration>
<sessionstate
mode="__mode__"
cookieless="false"
timeout="20"
sqlconnectionstring="data source=127.0.0.1;user id=<user id>;password=<password>"
server="127.0.0.1"
port="42424"
/>
</configuration>
I use this script:
# cd to the agent artifcats direcory (where the zip file exist)
cd $env:Agent_ReleaseDirectory
$fileToEdit = "web.config"
[Reflection.Assembly]::LoadWithPartialName("System.IO.Compression.FileSystem");
# Open zip and find the particular file (assumes only one inside the Zip file)
$zipfileName = dir -filter '*.zip'
$zip = [System.IO.Compression.ZipFile]::Open($zipfileName.FullName,"Update")
$configFile = $zip.Entries.Where({$_.name -like $fileToEdit})
# Read the contents of the file
$desiredFile = [System.IO.StreamReader]($configFile).Open()
$text = $desiredFile.ReadToEnd()
$desiredFile.Close()
$desiredFile.Dispose()
$text = $text -replace '__mode__',"stateserver"
#update file with new content
$desiredFile = [System.IO.StreamWriter]($configFile).Open()
$desiredFile.BaseStream.SetLength(0)
# Insert the $text to the file and close
$desiredFile.Write($text)
$desiredFile.Flush()
$desiredFile.Close()
# Write the changes and close the zip file
$zip.Dispose()
Before:
After (inside the zip file, without unzip and re-zip):
I was looking to do something similar, but found that there is a built-in task called File Transform [https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/file-transform?view=azure-devops] by Microsoft. With it, all you have to do is define a variable with the key in web.config if it is a simple substitute. If you need more involved transformation you can specify that too.
Since I had edge case, where I got 405 status on PUT and DELETE as seen here:
WebAPI Delete not working - 405 Method Not Allowed
which required me to change web.config file that is created only when project is released. So I needed to insert couple of lines of code in web.config like:
<modules>
<remove name="WebDAVModule" />
</modules>
and few more.
My answer is based on #Shayki Abramczyk one, I think it offers another, updated, take on this issue.
As his answer did not work fully for me, and for someone who is not professional in field of DevOps, rather programmer that wanted to automate the CI-CD stuff.
Issue I think is present nowadays is that line:
cd $env:Agent_ReleaseDirectory
is not navigating to proper folder. You still need to navigate to the folder and drop where your zip file is like so: cd _Your.Project-CI\drop
So start by adding another PowerShell component in your release pipeline like so:
And add following code to it:
# cd to the agent artifacts directory (where the zip file exist)
cd $env:Agent_ReleaseDirectory
cd _Your.Project-CI\drop
$fileToEdit = "web.config"
[Reflection.Assembly]::LoadWithPartialName("System.IO.Compression.FileSystem");
# Open zip and find the particular file (assumes only one inside the Zip file)
$zipfileName = dir -filter '*.zip'
$zip = [System.IO.Compression.ZipFile]::Open($zipfileName.FullName,"Update")
$configFile = $zip.Entries.Where({$_.name -like $fileToEdit})
# Read the contents of the file
$desiredFile = [System.IO.StreamReader]($configFile).Open()
$text = $desiredFile.ReadToEnd()
$desiredFile.Close()
$desiredFile.Dispose()
$contentToAdd1 = #'
<system.webServer>
<modules>
<remove name="WebDAVModule" />
</modules>
'#
#$text[3] = $text[3] -replace '<system.webServer>',$contentToAdd1
$text = $text -replace '<system.webServer>',$contentToAdd1
$contentToAdd2 = #'
<handlers>
<remove name="WebDAV" />
<remove name="ExtensionlessUrlHandler-Integrated-4.0" />
<add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="GET,HEAD,POST,PUT,DELETE,DEBUG" type="System.Web.Handlers.TransferRequestHandler" resourceType="Unspecified" requireAccess="Script" preCondition="integratedMode,runtimeVersionv4.0" responseBufferLimit="0" />
'#
# $text[4] = $text[4] -replace '<handlers>',$contentToAdd2
$text = $text -replace '<handlers>',$contentToAdd2
#update file with new content
$desiredFile = [System.IO.StreamWriter]($configFile).Open()
$desiredFile.BaseStream.SetLength(0)
# Insert the $text to the file and close
$desiredFile.Write($text)
$desiredFile.Flush()
$desiredFile.Close()
# Write the changes and close the zip file
$zip.Dispose()
Only thing that is left to do is to replace: cd _Your.Project-CI\drop with your project name e.g. cd _Weather.Front-CI\drop.

Automatically create i18n directory for VSCode extension

I am trying to understand the workflow presented in https://github.com/microsoft/vscode-extension-samples/tree/master/i18n-sample for localizing Visual Studio Code extensions.
I cannot figure out how the i18n directory gets created to begin with, as well as how the set of string keys in that directory get maintained over time.
There is one line in the README.md which says "You could have created this folder by hand, or you could have used the vscode-nls-dev tool to extract it."...how would one use vscode-nls-dev tool to extract it?
What I Understand
I understand that you can use vscode-nls, and wrap strings like this: localize("some.key", "My String") to pick up the localized version of that string at runtime.
I am pretty sure I understand that vscode-nls-dev is used at build time to substitute the content of files in the i18n directory into the transpiled JavaScript code, as well as creating files like out/extension.nls.ja.json
What is missing
Surely it is not expected that: for every file.ts file in your project you create an i18n/lang/out/file.i18n.json for every lang you support...and then keep the set of keys in that file up to date manually with every string change.
I am assuming that there is some process which automatically goes "are there any localize("key", "String") calls in file.ts for new keys not yet in file.i18n.json? If so, add those keys with some untranslated values". What is that process?
I have figured this out, referencing https://github.com/Microsoft/vscode-extension-samples/issues/74
This is built to work if you use Transifex for your translator. At the bare minimum you need to use .xlf files as your translation file format.
I think that this is best illustrated with an example, so lets say you wanted to get the sample project working after you had deleted the i18n folder
Step 1: Clone that project, and delete the i18n directory
Step 2: Modify the gulp file so that the compile function also generates nls metadata files in the out directory. Something like:
function compile(buildNls) {
var r = tsProject.src()
.pipe(sourcemaps.init())
.pipe(tsProject()).js
.pipe(buildNls ? nls.rewriteLocalizeCalls() : es.through())
.pipe(buildNls ? nls.createAdditionalLanguageFiles(languages, 'i18n', 'out') : es.through())
.pipe(buildNls ? nls.bundleMetaDataFiles('ms-vscode.node-debug2', 'out') : es.through())
.pipe(buildNls ? nls.bundleLanguageFiles() : es.through())
Step 3: Run the gulp build command. This will generate several necessary metadata files in the out/ directory
Step 4: Create and run a new gulp function to export the necessarry translations to the xlf file. Something like:
gulp.task('export-i18n', function() {
return gulp.src(['package.nls.json', 'out/nls.metadata.header.json', 'out/nls.metadata.json'])
.pipe(nls.createXlfFiles("vscode-extensions", "node-js-debug2"))
.pipe(gulp.dest(path.join('vscode-translations-export')));
}
Step 5: Get the resulting xlf file translated. Or, add some dummy values. I cant find if/where there is documentation for the file format needed, but this worked for me (for the extension):
<?xml version="1.0" encoding="utf-8"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
<file original="package" source-language="en" target-language="ja" datatype="plaintext"><body>
<trans-unit id="extension.sayHello.title">
<source xml:lang="en">Hello</source>
<target>JA_Hello</target>
</trans-unit>
<trans-unit id="extension.sayBye.title">
<source xml:lang="en">Bye</source>
<target>JA_Bye</target>
</trans-unit>
</body></file>
<file original="out/extension" source-language="en" target-language="ja" datatype="plaintext"><body>
<trans-unit id="sayHello.text">
<source xml:lang="en">Hello</source>
<target>JA_Hello</target>
</trans-unit>
</body></file>
<file original="out/command/sayBye" source-language="en" target-language="ja" datatype="plaintext"><body>
<trans-unit id="sayBye.text">
<source xml:lang="en">Bye</source>
<target>JA_Bye</target>
</trans-unit>>
</body></file>
</xliff>
Step 6: Stick that file in some known location, let's say /path/to/translation.xlf. Then add/run another new gulp task to import the translation. Something like:
gulp.task('i18n-import', () => {
return es.merge(languages.map(language => {
console.log(language.folderName)
return gulp.src(["/path/to/translation.xlf"])
.pipe(nls.prepareJsonFiles())
.pipe(gulp.dest(path.join('./i18n', language.folderName)));
}));
});
Step 7: Run the gulp build again.
The i18n/ directory should now be recreated correctly! Running the same build/export/translate/import/build steps will pick up any new changes to the localize() calls in your TypeScript code
Obviously this is not perfect, there are a lot of hardcoded paths and such, but hopefully it helps out anyone else who hits this issue.

Create new Simulink project through script

I want to create a new Simulink project through Matlab script.
Please show me how I can accomplish that through the use of syntax commands.
As am304 stated, it doesn't appear that simulinkproject can be directly created by a single command.
However, we can implement our own create simulinkproject method.
A simulinkproject is associated with a folder, and in this folder it stores all its project information the sub-folder, '.SimulinkProject'. In this subfolder, all information is stored in a file called Project.xml. If we create this sub-folder and initialize Project.xml, we have effectively created a simulink project, which can be open from the command line.
projectName = 'myProject';
s = strcat('<?xml version="1.0" encoding="UTF-8"?><project MetadataType="monolithic"> <EntryPoints Location="Root"/> <Files Location="Root"/> <ProjectData Location="Root"/> <Categories Location="Root"/> <Extensions Location="Root"/> <Info Location="ProjectData"> <Info Name="',projectName,'"/> </Info></project>');
path = 'C:\Users\Nathan\Documents\Simulink Files';
folderName = '.SimulinkProject';
filePath = strcat(path,filesep,folderName,filesep,'Project.xml');
mkdir(path,folderName);
file = fopen(filePath,'wt');
fprintf(file,s);
fclose(file);
myProj = simulinkproject(path);
Replace projectName with the desired name of the project and path with the desired location of the project.
MathWorks introduced a function to do this in R2017a. You can create a Simulink Project file (.prj) at the specified path by calling:
proj = slproject.create('projectpath\projectName');
Using the output object, proj, you can set up the project.
For more information, visit the MathWorks documentation.

How to write AssemblyVersion to file using MSBuild?

FinalEdit: Despite relative directories not working in the first post, it worked if I simply removed the $(MsBuildThisFileDirectory) from the Exec line.
Edit2: I added the new targets to the DefaultTargets. Which now runs them by default. However, timing was now off with the postbuild command. I added <Exec Command="call $(MsBuildThisFileDirectory)documentation\tools\GenerateDocumentation.bat" IgnoreExitCode="false" /> to the target, but it gives an error that C:\Users\my is not a valid batch file because of the space which is actually C:\Users\my program\documentation\tools\GenerateDocumentation.bat. Putting quotes around the path gives me error MSB4025 that Name cannot begin with $.
Edit: I have tried stijn's code and it works when I explicitly run it from the command line using /t:RetrieveIdentities, but for some reason it doesn't seem to run otherwise.
I have been using Doxygen to generate documentation for my source code, however, I would like to be able to do it automatically. I wrote a simple .bat script to run Doxygen with my desired config file and compile the output into a .chm help file, but I have been unable to change the revision number automatically in Doxygen.
I was attempting to simply update the config file by adding a new line to the config file with the new revision number using MSBuild, but I have been unable to get anything to print or even create a new file when none is present.
The code I have so far I have gotten from other similar questions, but I cannot seem to get it to work.
<ItemGroup>
<MyTextFile Include="\documentation\DoxygenConfigFile.doxyconfig"/>
<MyItems Include="PROJECT_NUMBER = %(MyAssemblyIdentitiesAssemblyInfo.Version)"/>
</ItemGroup>
<Target Name="RetrieveIdentities">
<GetAssemblyIdentity AssemblyFiles="bin\foo.exe">
<Output TaskParameter="Assemblies" ItemName="MyAssemblyIdentities"/>
</GetAssemblyIdentity>
<WriteLinesToFile File="#(MyTextFile)" Lines="#(MyItems)" Overwrite="false" Encoding="UTF8" />
</Target>
Encoding is wrong, it should be UTF-8
When working with items/properties, the % and # and $ must come right before the (, no spacing in between: %(MyAssemblyIdentitiesAssemblyInfo.Version)
MyAssemblyIdentitiesAssemblyInfo does not exist, you probably meant MyAssemblyIdentities
Look up how msbuild evaluates properties and items. Basically what it will do in your script is evaluate MyItems, but at that time MyAssemblyIdentities does not yet exist so is empty, and only afterwards the GetAssemblyIdentity gets executed. Fix this by enforcing correct evaluation order: put your items inside the target and make it depend on another target that creates MyAssemblyIdentities before evaluating your items.
To summarize:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="GetAssemblyIdentities">
<GetAssemblyIdentity AssemblyFiles="bin\foo.exe">
<Output TaskParameter="Assemblies" ItemName="MyAssemblyIdentities"/>
</GetAssemblyIdentity>
</Target>
<Target Name="RetrieveIdentities" DependsOnTargets="GetAssemblyIdentities">
<ItemGroup>
<MyTextFile Include="\documentation\DoxygenConfigFile.doxyconfig"/>
<MyItems Include="PROJECT_NUMBER = %(MyAssemblyIdentities.Version)"/>
</ItemGroup>
<WriteLinesToFile File="#(MyTextFile)" Lines="#(MyItems)"
Overwrite="false" Encoding="UTF-8" />
</Target>
</Project>
Note this will only work if you invoke msbuild in the directory where the script is, else the paths (documentation/foo) will be wrong. That could be fixed by using eg $(MsBuildThisFileDirectory)\bin\foo.exe)