VS_NEEDSNEWMETADATA in Visual Studio 2019/SSIS reading Timestamp-Column from PSDQL/ODBC - postgresql

I'm an Business Intelligence Developer. I'm developing for/maintaining a Microsoft BI Solution at our Company (SSIS,SSAS etc.). Recently I updated my Visual Studio to 2019 Enterprise (before that I was on 2015 Enterprise). Integration Services Extension Version 3.4. (also tested 3.2). With VS2015 everything was fine..
Part of our Data Warehouse is data coming from an PostgreSQL DB. I'm connecting to this using the official Postgres ODBC-Driver (32Bit, Unicode). I tried version 10.01 (old one, worked with VS15 without issues) and 12.01 (current one).
The Issue is the following: I opened the package I have the issues with. It warns me that the PSQL-ODBC-Datasource needs new metadata (nothing changed there nonetheless..). I can run the (unchanged) Package without any issues! Then I double-click the Source. A dialog pops up if I want to correct the Metadata of the Source according to the db it points to. Then the warning disappears. But If I run the package now it fails instantly telling me that the Source I just updated needs new Metadata.
I tracked the issue down to a minimal example. If I get data from a timestamp-column on the PSQL-ODBC-Datasource I allways get, immediately after execution of the package, an VS_NEEDSNEWMETADATA-Error in Visual Studio (SSIS) and the package won't run. There is no yellow Exclamation-Mark on the Source like it is usual. Everything seems fine on design time. I can even preview the output of the source in it's properties. If I select another column instead of the timestamp-column there is no issue. If I select the timestamp-column again it again gives the error on runtime. I also tried different Postgres servers with different versions as Source and different ODBC-Drivers.
If I set ValidateExternalMetadat to False on the source everything is running fine, but this can't be the solution as it worked flawlessly previous.
It seems like SSIS-Packages created with Visual Studio 2019 can't retrieve timestamp-columns from PSQL-ODBC-Sources if ValidateExternalMetadata is enabled.. This is driving me nuts!
Did anyone have this issue and find a solution?
Greetings
Edit: Further Investigation: It seems like VS2019 translates a PostgreSQL Timestamp now to dbTimeStamp2 instead of dbTimeStamp (like 2015 did) on the Output-Columns. But the Metadata also stored in the Package-Xml seems like it's telling that the columns are dbTimeStamp. This mismatch seems to be the Core-Issue why the Package is telling the the Metadata needs to be updated!?

OK I ran into the same problem here and it drove me crazy. I have the same setup and I got the exact same error messages.
Unfortunately I have no solution for this problem but at least I can tell you that it doesn't seem to be a isolated issue as I'm affected as well.
I overcame the problem for the time being by disabeling ValidateExternalMetadata but thats not a setup for production environments.
Thanks for this post though. I helped me not to feel like an idiot again. :D
Cheers
Daniel

Be careful with the casing. I had a view (as data source) in development, with two columns in uppercase.
Before creating the view in production, I decided to change to lowercase both columns, just because all was in lowercase and that two columns were the only things in uppercase. Then I created the view.
My ETL ran fine in DEV, but it didn't in PROD. Finally, I realized that the difference in casing was the problem.
When I changed the view in PROD, so that it was the same as in DEV, it worked smoothly.

Related

Problems with MS TFS 2018's Dashboards, Build Pipelines, and Icons

I am having some recent problems with TFS 2018 that have escaped my ability to troubleshoot. The application runs on a Hyper-V VM hosting Server 2019 and connects to a separate MS SQL 2016 database over on a separate Windows 2019 VM.
A few weeks ago we migrated our database server over to a new machine which, over the course of setting our existing TFS server up to use the new database, required us to run though the TFS setup wizard again.
Everything was fine for about a week when we started to have issues, specifically with the TFS web front-end we use. First we lost various icons on the webpage, with the browsers (Chrome, Firefox, etc). replacing them with rectangles:
A little while after that we lost our project Dashboards, and the whole dashboard page is just blank now. A little while after that, our WIP build/test automation feature also lost its management section of the site.
Other than these things not displaying, things generally "work" - the source control stuff functions, work items can be interacted with, etc. It's just that the interface is clumsier without the icons (which extends to every icon within every work item type, not just the banner I shared) and we can't get our automated test reports without the site's front-end for it. The latter is the real show stopper.
I spent some time troubleshooting and at best was able to figure out a maybe solution for the icon problem: https://social.msdn.microsoft.com/Forums/en-US/c1038468-9d94-473d-a020-254789e9a19b/tfs-2015-update-2-missing-icons?forum=tfsgeneral
This seemed to do the trick for just the icon problem, though some time later they disappeared and reappeared when people were refreshing pages. I'm still unsure if the re-failure was a fluke or not, as we rolled back the VM snapshot the changes were made on shortly after.
Using Chrome's developer tools, it seems like the lack of dashboard data is related to issues retrieving content on the host server for a cause we cannot determine.
Here is what shows up on the DevTool in Chrome for our main project's Dashboard:
What's interesting is the error claims widget.css is either not present or empty. Neither of these are the case as I can find the file and read data in it.
I recognize MIME types as a thing that shows up in IIS but I don't know what to do with the information. Should I be adding .css to the MIME Types list within IIS? Maybe that was set and the wizard reverted it?
Here's what shows up in the Builds section:
Thing is, I don't know what to do with this information. I found some vague hints online from people having similar issues with sites they were themselves coding (which stated the errors in question were red herrings), but this TFS front end is not something I've created and I had not any idea what to do with the information shared.
Does anyone have an idea of what might have gone wrong with the dashboards here? I have run out of ideas and can't figure out a different attack angle to approach this from.

STM32CubeIDE deletes files

I've got an issue with the CubeIDE from STM (Version 1.1.0).
I generated a Cube project, worked with it for 2 months, edited it,... no major problems. But suddenly the IDE deletes files like my main.cpp,..
I just did some adjustments, like adding another AD-Channel and generated the .ioc file with the MX-Editor.
Luckly I've got a backup on Bitbucket but I cannot work on ths project because I really need the adjustments I've done.
Is this a bug or am I too stupid to do some adjustments without deleting needed files?
Has anybody this problem too? Or a solution? I would be really thankful.

Accidentally published CompositeC1 with precompile on

I accidentally published Composite C1 from Visual Studio with "Precompile during publishing" when I first set it up and have now come to realize that several plugins do not work with precompile turned on. Unfortunately I have no idea how to reverse this. I've tried several things to no avail. I usually end up with a 500.0 error. If I go to the admin interface everything is working properly, and I can even preview the pages, but from the customer side all I get is a 500.0 error.
Of course I could republish from the Visual Studio project I have, but that project has the default template on it. I've tried copying what appears to be changed on the server to the original project but I end up with a 500 error. I've tried removing all the precompiled stuff on the server but I end up with a 500 error. I'm pretty lost.
At this point I'm willing to do anything. Is there a doc explaining how to copy the content to a newly downloaded copy, kind of like an upgrade doc but "side-grading"? I am using the most current version at the time of writing this 4.2 Update 1
Thank you
Solved use the PackageCreator suggested by wysocki's answer. It was not exactly straight forward to create the package. The errors can very cryptic at times. So here are some of the issues I encountered and how I got past them:
wysocki was correct to suggesting starting with a bare bones installation even though my original project was started with the Venus theme. I tried both ways.
I encountered issues with the MasterLayout and the Page Template Features which are currently not supported by the PackageCreator, although they may be supported in future versions. The solution was to add the missing elements to the fresh C1 project from the original C1 installation in the App_Data/PageTemplates and App_Data/PageTemplateFeatures folders.
A few times I ran into an issue where I added the same page or function twice to the PackageCreator. This was obviously my fault, but it should be mentioned that you can add the same site element twice which throws and error on import. Make sure you are careful to only getting one of each when creating the package to be exported.
If you are getting and error and it uses and Id like "4061397b-ee9e-4512-984d-f2b2d41eb654" I've found that it was very helpful to extract the zip file you are trying to import and then search the extracted folders for that Id. The lines with the Id usually have more information on them that will help you to identify exactly what content page or whatever the error is being generated from.
If you are using installed packages in your project like SimpleSearch for example, make sure you don't add it's functions to the PackageCreator. Install them separately.
I had an issue where I somehow had an element in the Data section that didn't have a "type". The error simply "The type cannot be empty". This was obviously tough to find so I suppose the lesson is, less is more. If you think your project might have a few elements that are unnecessary, don't add them at first and see if the project you import them into complains about missing them. It's much easier to troubleshoot missing elements than it is to figure out which element is causing the issue that you really didn't need.
Is there a doc explaining how to copy the content to a newly downloaded copy
You can try and export content / templates / anything else related to a "package" via Composite.Tools.PackageCreator.
Its latest version gives you quite a lot of flexibility in what you can export ("package") - please its user manual.
Once the package is made and downloaded, you can go on and install it on a new website. If you export / import a lot of content / templates etc, it makes sense to have a "Bare Bones" site as the new one.
All the starter sites like "Neptune", "Venus" are installed as a bunch of packages in specific order during Composite C1's initial setup.

Changes in Umbraco CMS does not update at front-end instantly

I have an issue in updating contents in Umbraco. Whenever I update something in Umbraco, I have to wait at least one hour, sometimes 12 hours to see the changes at front-end.
The only way to see the changes immediately at front-end is "empty the connection string value umbracoDbDSN and refresh the page, then put the connection string back and refresh the page". I have to do everything I update something in CMS.
Do you guys have any idea what is happening here? Thanks.
The problem was. Umbraco was configured to run on load balanced servers on our old servers. I had to turn it off on the new server.
<distributedCall enable="false"> in umbracoSettings.config
What version are you running? When v5 first came out, I had a big problem with that (and solved it like you by touching the web.config to force a reset. Hopefully you are not using v5 (as its been discontinued and has extreme performance issues).
I have not had that problem in any v4.x versions that I can remember; changes should show up instantly after you republish.
Are you running in a standard configuration? Using a webfarm by anyt chance?
Is the ~/App_Data/umbraco.config file being written to on publish? This is the XML cache file that is used in displaying you website.
When you publish a node, the data is serialized into XML, stored in a database table and then written out to the umbraco.config cache.
This could be some kind of permissions issue, if umbraco doesn't have rights to read/write the file. Or you could have a corrupt dll that just isn't writing to it correctly. Or perhaps it's writing it out just fine, but your server is caching you pages in a weird way. Either way, I'd take a look first at the umbraco.config and make sure the data is being written to it on publish.

Is AnkhSVN any good?

I asked a couple of coworkers about AnkhSVN and neither one of them was happy with it. One of them went as far as saying that AnkhSVN has messed up his devenv several times.
What's your experience with AnkhSVN? I really miss having an IDE integrated source control tool.
Older AnkhSVN (pre 2.0) was very crappy and I was only using it for shiny icons in the solution explorer. I relied on Tortoise for everything except reverts.
The newer Ankh is a complete rewrite (it is now using the Source Control API of the IDE) and looks & works much better. Still, I haven't forced it to any heavy lifting. Icons is enough for me.
The only gripe I have with 2.0 is the fact that it slaps its footprint to .sln files. I always revert them lest they cause problems for co-workers who do not have Ankh installed. I don't know if my fears are groundless or not.
addendum:
I have been using v2.1.7141 a bit more extensively for the last few weeks and here are the new things I have to add:
No ugly crashes that plagued v1.x. Yay!
For some reason, "Show Changes" (diff) windows are limited to only two. Meh.
Diff windows do not allow editing/reverting yet. Boo!
Updates, commits and browsing are MUCH faster than Tortoise. Yay!
All in all, I would not use it standalone, but once you start using it, it becomes an almost indispensable companion to Tortoise.
I always had stability issues with AnkhSVN. I couldn't switch everyone to Subversion where I work without an integrated solution.
Thank goodness for VisualSVN + TortoiseSVN.
VisualSVN isn't free, but it is cheap, and works a treat.
I tried version 1, and it was unreliable to say the least. I can't say anything about 2.0.
If you can afford it, the one I use, VisualSVN, is very good and uses TortoiseSVN for all its gui, except for the specialized things related to its VS integration.
#pilif: AnkhSVN maintains an in-memory state of the working copy, which is invalidated/updated by Visual Studio events (ie you edit/change a file) and AnkhSVN events (ie you commit/update/revert/etc)
Whenever the working copy is changed from outside Visual Studio (by editing with another tool, or by using another Subversion client), you will have to refresh AnkhSvn using the Refresh command we provide.
The other thing that happens when you delete a file in a project with TortoiseSvn for example, is that it remains listed in the project file, and you will have to remove it there seperately (and then commit the project file as well).
Copy/Pasting parts of my own Blogpost, as I switched from Ankh to VisualSVN:
Why did I switch? Because i was a bit unhappy with the overall stability of Ankh, since it has some problems actually tracking Solution changes. VisualSVN is “just” a TortoiseSVN Frontend, which means it leaves all the “heavy lifting” to a third-party tool that a) is installed on most Workstations anyway and b) that’s been tested and used by such a wide audience, it’s really rock-solid.
Now, AnkhSVN is certainly not a bad product, and the people behind it are serious about what they are doing, but having long-deleted files still in my SVN or getting the “Please Cleanup your solution” message get’s annoying after some time, but my biggest gripe is the property window. It’s nice that there is a nice window with Radio Buttons asking me which property I want to add. Unfortunately, there is no way to manually enter a property.
Edit: That was for AnkhSVN 1.x. In the meantime, it was updated to 2.x and much improved. I use it in production on a system where I don't have VisualSVN and it works extremely well now.
I had no problems with v1, but I was warned not to use it. I've been using v2 for a while, and I've had no problems with it. I still keep a backup of the repository though...
I started with AnkhSvn and then moved on to VisualSvn. I have my own gripes with VisualSvn but its far less trouble compared to Ankh. I'm yet to try the new version of Ankh which they say is a complete rewrite and had inputs from Microsoft dev team as well.
I've been using both the newest version of Ankh SVN and Tortoise on a project at home. I find them to both be very good with a caveat.
I've found that both SVN tools have at times failed to keep up with my file/folder renaming and moving resulting in it thinking that a perfectly good file needs to be deleted on the next commit. This is probably down to me misusing SVN in some way but TFS at work does not have this problem.
I tried AnkhSVN (1.0.3, just 4 months ago), and it did not work the way I wanted it to (i.e. needed to select things in the browser window instead of based on active file). I ended up making some macros that utilize TortoiseSVN that work much more like what I expected.
I've been very happy with using TortoiseSVN via explorer and my macros inside the IDE.
Earlier on (like 2 years ago when I last tried), AnkhSVN and Tortoise used in parallel with the same working copy caused some kind of working copy corruption where Ankh and Tortoise somehow lost track of the state the other tool left the working copy in.
It was as if one of the tools stored additional metadata not contained in the working copy and was reliant on that being correct.
The problems showed themselves by Ankh (or Tortoise) insisting on files being there which weren't, on files being changed which weren't and on files not being changed which were (and thus unable to commit).
Maybe this has been fixed since, but I thought I'd better warn you guys.
About a year ago me and a buddy used AnkhSVN for a project... several commits later while moving namespaces around, it broke the SVN repository. Broke as in, the last commit we did got corrupted, and we couldn't commit anymore.
After that we used TortoiseSVN and did the namespace moving manually, it just... worked. If you're only working on base class libraries you could always try using SharpDevelop instead (that integrates with TortoiseSVN).
I do hope they did fix AnkhSVN now though because IDE integrations always rock... when they work.
#mcintyre321
I've found that both SVN tools have at times failed to keep up with my file/folder renaming and moving resulting in it thinking that a perfectly good file needs to be deleted on the next commit.
A move or rename operation results in an delete and 'add with history' at subversion level.
TortoiseSvn shows this as:
originalFile deleted
newFile added (+)