Is there mlflow REST api to hard delete experiments, runs? - rest

Mlfow exp delete api does soft delete and when you create experiment with that name, it gives error RESOURCE_ALREADY_EXISTS.
Is there any way to delete experiment permanently through api?
https://www.mlflow.org/docs/latest/rest-api.html#delete-experiment
There is similar question here, How Do You "Permanently" Delete An Experiment In Mlflow?
where answers are to run delete sql queries directly by connecting to backend DB which i want to avoid.

Related

It is possible to fetch data from the repository using typeorm for mongodb

I'm making an api in which I separate my requests by responsibilities, services, repository and routes, I'm using typeorm to connect to the database, and I'm trying to implement unit tests to test my application so I have to unclip my code from the typeorm of so that my tests are not depending on it. However, at the time of decoupling, the typeorm is no longer able to connect with the bank, which is mongodb, returns the following error: ConnectionNotFoundError: Connection "mongo" was not found.
I've been reading about it and from what I understood the typeorm wouldn't give full support to the mongo, that's why it failed. I would like to confirm with you if that would be the case and what would be the best alternative to solve this.
MongoDB is supported by TypeORM (although I have not used it personally myself).
You can follow a short tutorial here.
What is not supported for Mongo is the typeorm "Migration" feature. For example see typeorm issue 6695. Migration is an advanced topic and you probably do not need it. But if you do, a workaround is on stackoverflow answer here.
But this Migration issue has nothing to do with your error. Almost certainly, your getConnection() call or ormconfig.json are incorrect. Start a with simple project based on the TypeORM Mongo tutorial, nothing else, verify you can connect to MongoDB, and work up from there.

Use AWS Amplify and App Sync with existing Node Server using Mongodb

Currently, I'm developing a native application using React-Native. I've decided to go with AWS Amplify because of it's real time updates as well as its authentication.
I also have a Web Application that runs on a Node.js with Epxress server. This web application connects to a Mongo database.
My big problem is that I would like to have all of my aws amplify queries run to my existing MongoDb instead of a new dynamoDb database which is provided with AWS AppSync, but unfortunately I dont know where to start. This is especially helpful in adding authentication easily in my existing web application as well.
My first idea was to just create all my API endpoints in a new node js server and have app sync call to these API end points, but I'm not sure how to implement calling end points on an existing server (and this seems kind of counter intuitive to the 'serverless' idea)
My other idea came from this: Can AWS App-Sync be used without dynamoDB
This states to use AWS Lambda to 'pipeline' my data to the existing mongodb, but I'm not really sure what that entails.
TL;DR - I would like to be able to query an existing Mongodb instead of using DynamoDb when using AWS Amplify with AppSync.
I hope this is clear enough and doesn't sound like I'm rambling. Thanks in advance!
I would suggest using either an HTTP datasource to connect to your MongoDB backend or a Lambda function. Here are a couple getting started tutorials for both:
https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-http-resolvers.html
https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-lambda-resolvers.html
If you go the Lambda route, then you can leverage the new #function feature of the GraphQL Transformer in the Amplify CLI: https://aws-amplify.github.io/docs/cli/graphql#function

Can I force delete an AWS CloudFormation stack that is In Progress of Rollback

An AWS CloudFormation rollback (e.g., UPDATE_ROLLBACK_IN_PROGRESS) has been in progress forever, like over an hour and a half. I want to delete the stack altogether or force stop any activity. Is this possible?
Thanks!
Another common cause of blocked stack updates/rollbacks is errors in ECS::Service resource updates: it doesn't look like that is currently detected (in some cases?). Cloudformation is waiting for the service event for the service reaching a steady state, so simply updating the service to something that works (e.g. desired tasks to 0) will unblock it. Try to get the state back to what Cloudformation expects before sending more updates to avoid problems, though.
I guess your stack resources are changed or deleted by outside.
You can find official guide as below.
Manually sync resources so that they match the original stack's template, and then continue rolling back the update. For example, if you manually deleted a resource that AWS CloudFormation is attempting to roll back to, you must manually create that resource with the same name and properties it had in the original stack.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html#troubleshooting-errors-update-rollback-failed
or (as #talentedmrjones said)
To fix the stack, contact AWS customer support.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html#troubleshooting-errors-nested-stacks-are-stuck
In my case, I can stop same situation via re-creating deleted resource.
In my case it is an EC2 security group that cannot be deleted because it is referenced from another EC2 security group.
When dealing with a custom resource it is possible to construct a mocked up version of the return url.
The easiest way to do this is to grab the url which was used during the create. If you can get your hands on it, replace the section after the last %2F with the "Client Request Token" which you can get from your event log for the cloudformation.
If not, then here's the format of the url you'll have to construct.
https://{region}.console.aws.amazon.com/cloudformation/home?region={region}#/stacks?filter=active&tab=events&stackId={stack arn}%2F{stack name}%2F{client request token}
Run that url as a get and it will cause the resource to fail rollback or delete.
You can try to delete the resources and then the update rollback will complete successfully.
Sometimes this will occur if your user role is missing permissions to delete roles. This can be tested by trying to manually delete roles or users that have been created by the CloudFormation stack.
I had something like this happen once, and the stack seemed stuck forever in UPDATE_ROLLBACK_IN_PROGRESS status. I'd recommend submitting a ticket to AWS support. That was the only way I was able to resolve it.
Was able to delete mine by manually deleting everything via AWS dashboard. I ended up having a couple dangling roles that just needed deletion.
I meet the same problem.
The console told me some resource depends on another, so can't be deleted. Under that state, rollback in unavailable.
I just delete the whole VPC and the resources in that VPC.
Because cloudformation will retry to delete resource every 10-20 min. So when it retry, it will find the resource have already been deleted, and it just skip the deletion and everything is smooth after that.
Yes, use this command to delete stacks stuck in 'DELETE_IN_PROGRESS' state.
You can easily run this in AWS CloudShell also.
Go to Lambda Function->Monitor->CloudWatch Logs. Look for Log where "RequestType" is "Delete" and Copy the necessary fields to below command
curl -H 'Content-Type: ''' -X PUT -d '{"Status": "SUCCESS","PhysicalResourceId": "Add your physical resource ID", "StackId": "Add your StackId","RequestId": "Add your RequestID","LogicalResourceId": "LambdaFunction"}' 'Add your ResponseURL Here'
Example:
curl -H 'Content-Type: ''' -X PUT -d '{"Status": "SUCCESS","PhysicalResourceId": "cutomRes-LambdaFunction-1NC1ORF", "StackId": "arn:aws:cloudformation:us-east-1:3343:stack/cutomRes/f52a-11eb-b5df-0a5c2cc1","RequestId": "d70931a2-364b-413e-a2","LogicalResourceId": "LambdaFunction"}' 'https://cloudformation-custom-resource-response-useast1.s3.amazonaws.com/arn%3Aaws%/cutomRes/f5466f6Expires=7200&X-Amz-Credential=AKIA6L7Q4OWT3GW5BT7K%2F20210330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1db1f83f'
Do Note that, example contains URL that mmaybe modified to not work for security purposes. It is for demonstration purposes only.
You will need to investigate why exactly the rollback is taking so long (e.g., if it's due to a missing resource modified outside of the CloudFormation stack, or a Custom Resource that failed to return the expected signals).
I went to the stack resources tab and checked why some of them couldn't be deleted then I deleted them manually first.
Usually, it works with just a quick refresh.
We need to go to the resources section and check which resource it is trying to delete. Go to that resource and check why Cloud formation is not able to delete that resource.
What we can do is try deleting the resource manually and check the error or dependency. Fix that and the stack will again then continue. Depending on you stack design and dependencies you might have to delete manually or fix manually multiple resources.
Check this for more details :
https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-stack-stuck-progress/

Get-AzureRmResourceGroupDeployment lists machines I cannot see in the web interface

I'm tasked with automating the creation of Azure VM's, and naturally I do a number of more or less broken iterations of trying to deploy a VM image. As part of this, I automatically allocate serial hostnames, but there's a strange reason it's not working:
The code in the link above works very well, but the contents of my ResourceGroup is not as expected. Every time I deploy (successfully or not), a new entry is created in whatever list is returned by Get-AzureRmResourceGroupDeployment; however, in the Azure web interface I can only see a few of these entries. If, for instance, I omit a parameter for the JSON file, Azure cannot even begin to deploy something -- but the hostname is somehow reserved anyway.
Where is this list? How can I clean up after broken deployments?
Currently, Get-AzureRmResourceGroupDeployment returns:
azure-w10-tfs13
azure-w10-tfs12
azure-w10-tfs11
azure-w10-tfs10
azure-w10-tfs09
azure-w10-tfs08
azure-w10-tfs07
azure-w10-tfs06
azure-w10-tfs05
azure-w10-tfs02
azure-w7-tfs01
azure-w10-tfs19
azure-w10-tfs1
although the web interface only lists:
azure-w10-tfs12
azure-w10-tfs13
azure-w10-tfs09
azure-w10-tfs05
azure-w10-tfs02
Solved using the code $siblings = (Get-AzureRmResource).Name | Where-Object{$_ -match "^$hostname\d+$"}
(PS. If you have tips for better tags, please feel free to edit this question!)
If you create a VM in Azure Resource Management mode, it will have a deployment attached to it. In fact if you create any resource at all, it will have a resource deployment attached.
If you delete the resource you will still have the deployment record there, because you still deployed it at some stage. Consider deployments as part of the audit trail of what has happened within the account.
You can delete deployment records with Remove-AzureRmResourceGroupDeployment but there is very little point, since deployments have no bearing upon the operation of Azure. There is no cost associated they are just historical records.
Querying deployments with Get-AzureRmResourceGroupDeployment will yield you the following fields.
DeploymentName
Mode
Outputs
OutputsString
Parameters
ParametersString
ProvisioningState
ResourceGroupName
TemplateLink
TemplateLinkString
Timestamp
So you can know whether the deployment was successful via ProvisioningState know the templates you used with TemplateLink and TemplateLinkString and check the outputs of the deployment etc. This can be useful to figure out what template worked and what didn't.
If you want to see actual resources, that you are potentially being charged for, you can use Get-AzureRmResource
If you just want to retrieve a list of the names of VMs that exist within an Azure subscription, you can use
(Get-AzureRmVM).Name

How to insert data into my SQLDB service instance (Bluemix)

I have created an SQLDB service instance and bound it to my application. I have created some tables and need to load data into them. If I write an INSERT statement into RUN DDL, I receive a SQL -104 error. How can I INSERT SQL into my SQLDB service instance.
If you're needing to run your SQL from an application then there are several examples (sample code included) of how to accomplish this at the site listed below:
http://www.ng.bluemix.net/docs/services/SQLDB/index.html#run-a-query-in-java
Additionally, you can execute SQL in the SQL Database Console by navigating to Manage -> Work with Database Objects. More information can be found here:
http://www.ng.bluemix.net/docs/services/SQLDB/index.html#sqldb_005
s.executeUpdate("CREATE TABLE MYLIBRARY.MYTABLE (NAME VARCHAR(20), ID INTEGER)");
s.executeUpdate("INSERT INTO MYLIBRARY.MYTABLE (NAME, ID) VALUES ('BlueMix', 123)");
Full Code
Most people do initial database population or migrations when they deploy their application. Often these database commands are programming language specific. The poster didn't include the programming language. You can accomplish this two ways.
Append a bash script that would call your database scripts that you uploaded. This project shows how you can call that bash script from within your manifest file as part of doing a CF Push.
Some languages like offer a file type or service that will automatically get used to populate the database on initial deploy or when your migrate/synch the db. For example Python Django offers a "fixtures" file that will automatically take a JSON file and populate your database tables