If I write the following url in the browser, I get a CSV file with weekly data from Google Trends:
www.google.com/trends/trendsReport?q=SearchTerm&export=1
In the past, this file would have been the same as the one obtained by manually clicking the "export as CSV" button on the Google Trends page. But now, instead, by using this button I get a data file with monthly data.
Does anyone know whether it is possible to obtain this monthly file by using an url similar to the above-mentioned one?
You could use date parameter with your request
www.google.com/trends/trendsReport?q=SearchTerm&export=1&date=01/2010 37m
Some pages recommend to use more than 36 months, but I tried with 140 months and always returns weekly data. Check this page for more reference, https://github.com/GeneralMills/pytrends
Also, you could check how to call this url
https://www.google.com.mx/trends/api/widgetdata/multiline/csv?req=%7B%22time%22%3A%222004-01-01%202016-10-07%22%2C%22resolution%22%3A%22MONTH%22%2C%22locale%22%3A%22es-419%22%2C%22comparisonItem%22%3A%5B%7B%22geo%22%3A%7B%7D%2C%22complexKeywordsRestriction%22%3A%7B%22keyword%22%3A%5B%7B%22type%22%3A%22BROAD%22%2C%22value%22%3A%22pope%22%7D%5D%7D%7D%5D%2C%22requestOptions%22%3A%7B%22property%22%3A%22%22%2C%22backend%22%3A%22IZG%22%2C%22category%22%3A0%7D%7D&token=APP6_UEAAAAAV_kTww_1wNWLYGrce91gQBIxkGPV4lGg&tz=300
This return a monthly data, but it appear you need to use a token
decoded url:
https://www.google.com.mx/trends/api/widgetdata/multiline/csv?req={"time":"2004-01-01 2016-10-07","resolution":"MONTH","locale":"es-419","comparisonItem":[{"geo":{},"complexKeywordsRestriction":{"keyword":[{"type":"BROAD","value":"pope"}]}}],"requestOptions":{"property":"","backend":"IZG","category":0}}&token=APP6_UEAAAAAV_kTww_1wNWLYGrce91gQBIxkGPV4lGg&tz=300
Related
I am working on a pipeline which executes oAuth2 flow in order to access REST API json data. Once I have the bearer token I am executing a request which returns the following structure:
As you can see, since the response is quite large, there's paging enabled and as part of the response I get a link to the next page. In order to get to that resource I need to also present MS-ContinuationToken in the headers. So, this is how I basically do it in the config of the Copy activity that I use to get the data from the REST endpoint:
and the issue here is that I only get the first 2000 rows and the next page(s) don't seem to be visited at all. Pipeline executes successfully and only the first 2000 items are fetched.
NOTE: continuationToken and links.next.headers.value have the exact same values from the initial response.
Even if you fix the other issue you’ll have an issue with the “next” URL not including “v1”. This is a known issue in the partner center api team. I’ve escalated it pretty high. But they don’t want to break backwards compatibility by changing the “next” URI to include the v1 or to be relative. They are considering other options but I wouldn’t hold your breath.
I would ditch the idea of using data factory and instead write a .NET console app using the partner center SDK
(You might think to paginate manually with loops etc but the Copy activity doesn’t return eg the http headers, so you will need a complex set up to somehow store the data in a data store and be able to look up the last page in order to get the continuation token. I couldn’t figure it out)
I've got a database containing offline conversions (email, phone, name, purchase_amount, etc). I can export this database in .csv or .xls and I can also email this file on a daily basis to a Gmail account.
As Zapier has a Google Sheet to "Facebook offline event" API, I tried this workflow with Zapier.com:
Export my database in .xls: OK
Mail it to my Gmail account as an email attachment: OK
Grab the attachment and upload file to Google Drive using Zapier: OK
This is the part where I'm in trouble: I want to copy the content of the .xls file that is on Google Drive to a new Google Sheet. I can't figure out how to do this in Zapier.
Finally, on every new spreadsheet created or new row added (depending on how I configure the Zap) , pushing the data to Facebook API.
I'm not a developer so I want to avoid coding if possible. I tought I could easily do it with zapier but it seems that working with data inside a file is not so easy.
Any help would be much appreciated.
Thank you,
Best regards,
Tim.
If it were me I would look into the scripting capabilities of Google Sheets to try and achieve this, having your code execute from a single place eliminates other possible points of failure. That said, I have put together a somewhat hacky, code free solution that should set you up to do what you are looking to achieve. I break it down step by step below:
Step 1: Export database as .csv file. I could only get this to work with .csv files and not .xlsx files. There may be the ability to do so but it would require further trial and error.
Step 2: Mail it to your Gmail account where I assume there is a Zap which triggers to upload the attachment to your drive account automatically.
Step 3: Setup a second Zap that is connected to your Gmail account
that triggers when you receive an email with an attachment.
Step 4: Isolate the attachment file from the results of the triggered Zap and use it as input for the following formatter action step.
Step 5: Setup your formatter action step using the text option. Within the formatter template select trim white space and use the attachment, isolated from the trigger step, as its input. See example photo here.
Step 6: Setup your final step which is the create Google Sheet function of the Google Sheets Zap. Enter a title for your new sheet, it will probably need to be a unique value I used the attachment ID from step one as my title but you can set it to whatever you would like. In the headers section type =IMPORTDATA("") . Between the two quotation marks place the output of the previous formatter step and then run the Zap. See example photo here.
Explanation: When Zapier catches the attachment file from your inbound email it seems to be stored as raw data. Given this we cannot simply dump this information into a spreadsheet as it would be unreadable. However it seems Zapier has a method for converting this raw data through the endpoint https://zapier.com/engine/hydrate. When we input the raw attachment data into the formatter step Zapier provides a link pointing to the URL for converting the data into its original format. We take this URL and using the Google worksheet function IMPORTDATA() we are able to import the file using Zapier's file conversion engine. Now that the data is in your new sheet you can set up an additional Zap to do something with it. Also note that the Zap to upload the attachment to your Google Drive is not necessary with this setup. That said if you are looking to keep backups of your data then keep it on otherwise you may have the opportunity to save yourself some zaps.
Hope this helps!
Many thanks for your awesome reply. I also tried the "trim whitespace" to get the data back. I only missed the "importdata" function which is super powerful. Indeed it only works with .csv. With .xls file, importdata gives the source code of xls file which is useless.
I ended with 2 zaps:
Grab Gmail attachment, upload to Google Drive (for backup & monitoring) and create new spreadsheet
Send Facebook offline conversion when new spreadsheet is added (filter: only continue when file name is xxxx), lookup spreadsheet row (I took one column that has the same value for each row) and finaly I could match my columns with the Facebook API.
Separate issue raised with Google by st...#ditoweb.com here (issue 2391)
We have an iframe-based spreadsheet form on a website (http://www.monarch-equestrian.co.uk/brochurerequest.html) which is set to generate emails with attached PDF's to respondents.
Over the past few days, this has been generating an error notification - "You do not have permission to call getActiveSpreadsheet". The script, triggered on submission, has been working perfectly - however the spreadsheet had grown to 1387 rows so we archived it and deleted most of the records in the original (in case we had reached the limit allowed and so we wouldn't have to create a new spreadsheet etc).
The question is, even though the spreadsheet is presumably now within data limits and all the settings are unchanged, why are we still getting the error?
According to the reported issue noted in the question, this was fixed Feb 25, 2013.
I want to post some data to the server, and in response, I want to create a CSV file, with application/excel as the MIME Type (recently recognized as Internet Media Type), to force the browser to open the generated CSV file in Microsoft Excel. However, I also want to prevent user from re-submitting the same info (re-posting the form) by any accident as the result of refreshing the page.
With simple CRUD operations, I use Post/Redirect/Get pattern, so that any further refreshing will only send HTTP Get Request to the server, without any parameter, thus not changing server's state (Idempotence).
What is the recognized pattern for stopping user from re-submitting (re-posting) the same info to the server, when the response is not a page, but a file?
Any idea?
The Post/Redirect/Get pattern is an answer to a browsing event.
Here, there is no browsing action (the form submission only open a 3rd party app, i.e excel), and so any kind of browsing related pattern will be useless.
I suggest you use both a server side trace of the initial submission (with a unique token maybe), so you can prevent the file generation, and an easy to write client side script like <form onsubmit="this.onsubmit = function(){ return false ; }">
I can offer you one other solution.
Take hash (MD5/SHA256 ..) of your submitted data. The hash will be [fairly] unique.
Put it in list in a session with a time limit, say 5 minutes.
Even your user submit same data. Hash will be same and you can give error message to your user.
If different users can post same data, you can also hold user information in the list. And give error message according to user.
I'm trying to use Google Analytics API to retrieve data for my custom segments.
They say here http://code.google.com/apis/analytics/docs/gdata/gdataReferenceAccountFeed.html#accountResponse that all the segments are listed in the account feed response. But mine doesn't contain them.
Has anyone else had this problem and know a way to get the data out from them?
Thanks,
Beth
Make sure you add GData-Version: 2 in your header.