As with others who have posted in the past, I cannot figure out to download a csv.gz file from a URL in Python 3.7. I see posts but they only post a 2kb file.
I am a 100% newbie using Python. What follows is the code for one file that I am trying to obtain. I can't even do that. The final goal would be to request all files that start with 2019* using python. Please try the code below to save the file. As others stated, the file is just a name without the true content - Ref: Downloading a csv.gz file from url in Python
import requests
url = 'https://public.bitmex.com/?prefix=data/trade/20191026.csv.gz'
r = requests.get(url, allow_redirects=True)
open('20191026.csv.gz', 'wb').write(r.content)
Yields:
Out[40]:
1245
I've tried "wget" and urllib.request along with "urlretrieve" also.
I wish I could add a screenshot or attach a file. The file created is 2kb and not even a csv.gz file. But the true file that I can download from a web browser is 78mb. The file is 20191026.csv.gz not that it matters as they all do the same thing. The location is https://public.bitmex.com/?prefix=data/trade/
Again, if you know of a way to obtain all the files using a filter such that 2019*csv.gz would be fantastic.
You are trying to download the files from https://public.bitmex.com/?prefix=data/trade/.
To achieve your final goal of download all the files starting from 2019* you have to do in 3 steps
1) you read the content of https://public.bitmex.com/?prefix=data/trade/
2) convert the content into an list, from that filter out the file names which starting from 2019.
3) from the result list try to download the csv using the example which you referring.
Hope this approach will help you
Happy coding.
Related
I'm sorry if this doesn't have enough information. I don't typically ask for help online like this.
I'm using DITA Open Toolkit 3.4 on Windows. I generated a plugin called "vcr2" using Jarno's (very excellent and helpful) PDF Plugin Generator and then made a handful of customizations. The plugin uses the pdf2 plugin as a base. When I try to use the vcr2 plugin, my images are not working. I've tracked the problem down to malformed image filenames in the image's href attribute.
For example:
In my source file (a DITA Task), the markup for one of my images looks like this:
<image href="MyRemindersChooseReminder.png"/>
If I run a transform with the pdf2 plugin, the images work fine. In the merged stage1.xml file in the Temp folder, the XML for that same image looks like this:
<image class="- topic/image " href="df2d132af27436c59c5c8c4282e112d62bec8201.png" placement="inline" xtrc="image:1;10:66" xtrf="file:/V:/Vasont/Extract/t12340879-minimal/t12340879.xml"/>
It is processed into a file Topic.fo, and looks like this:
<fo:external-graphic
src="url('file:/V:/Vasont/Extract/t12340879-minimal/MyRemindersChooseReminder.png')"/>
Everything works fine and the image looks fine.
If I run the same file through my 'vcr2' plugin, which just calls the same pdf2 plugin with some overrides, all the images get broken:
stage1.xml
<image class="- topic/image " href="df2d132af27436c59c5c8c4282e112d62bec8201.png" placement="inline" xtrc="image:1;10:66" xtrf="file:/V:/Vasont/Extract/t12340879-minimal/t12340879.xml"/>
Topic.fo
<fo:external-graphic
src="url('file:/V:/Vasont/Extract/t12340879-minimal/df2d132af27436c59c5c8c4282e112d62bec8201.png')"
/>
As I track this down further, it appears that somewhere in the map-reader Ant task, this filename gets changed to that cryptic string of pseudo-hexadecimal. I think later on it's supposed to be changed back or resolved to a complete URI or something.
So, the two-part question is: Why does Open Toolkit change my filenames, and what's supposed to change them back?
DITA-OT's preprocess uses hashes for temporary filenames because it allows the code to not deal with directory structures. This enables preprocess to work in so-called "map-first" mode, where it first processes all DITA map resources and only then starts to process DITA topic and image resources.
The preprocess has a step called clean-preprocess that can rewrite the temporary file names to match source resource files names. However, this rewrite operation is disabled for PDF output because the original file names are not used for anything in that output type.
I have a password protected zip file with 3 folders PFC, STA and SYS. The zip file when loaded for the first time reads all the text files in the PFC folder and display the data onto the screen. I perform some operations on the screeen which generates new files and they are added into the STA and SYS folders and the same initial zip file is updated.
However, when I try to load the same zip file which now should have the all the initial files and folders along with the newly created files in it, it gives me a Wrong CRC Value error. I dont know what am I doing wrong.
We have tried to reproduce the issue by preparing a sample with your scenario. But, we are facing another issue related to password which we will fix in our upcoming release. The sample can be downloaded from the below link .
Sample link : https://www.syncfusion.com/downloads/support/directtrac/general/ze/Password-1299200154.zip
Kindly modify the sample to reproduce the “CRC value error” issue and update us along with input documents. It will help us to analyze further on this and provide you a solution at earliest.
Note : I work for Syncfusion.
You can send your sample or confidential information in mail to, support#syncfusion.com
Regards,
Mohan.
Product CSV created in var folder but it's not getting the download. Or CSV file shows 302 error.
I tried all the solutions given for not working export. is there any other way?
1.expected result: It should download a CSV file
2.actual result: It only generates a file in the var folder but not getting a download.
You have to remove deprecated "modules_disable_output" values from core_config_data,
Before run below query please take backup of core_config_data table.
DELETE FROM core_config_data WHERE path LIKE
"advanced/modules_disable_output/%";
And clear cache
Hope it will work for you.
I am trying a proof of concept here:
Using Chaquopy 4.0.0 (I use python 2.7.15), I am trying to write a string to file in a specific folder (getFilesDir()) using Python, then reading in via Android.
To check whether the file was written, I am checking for the file's length (see code below).
I am expecting to get any length latger than 0 (to verify that the file indeed has been written to the specific location), but I keep getting 0.
Any help would be greatly appreciated!!
main.py:
import os.path
save_path = "/data/user/0/$packageName/files/"
name_of_file = raw_input("test")
completeName = os.path.join(save_path, name_of_file+".txt")
file1 = open(completeName, "w")
toFile = raw_input("testAsWell")
file1.write(toFile)
file1.close()
OnCreate:
if (! Python.isStarted()) {
Python.start(new AndroidPlatform(this));
File file = new File(getFilesDir(), "test.txt");
Log.e("TEST", String.valueOf(file.length()));
}```
It's not clear whether you've based your app on the console example, so I'll give an answer for both cases.
If you have based your app on the console example, then the code in onCreate will run before the code in main.py, and the file won't exist the first time you start the activity. It should exist the second time: if it still doesn't, try using the Android Studio file explorer to see what's in the files directory.
If you haven't based your app on the console example, then you'll need to execute main.py manually, like this:
Python.getInstance().getModule("main");
Also, without the input UI which the console example provides, you won't be able to read anything from stdin. So you'll need to do one of the following:
Base your app on the console example; or
Replace the raw_input calls with a hard-coded file name and content; or
Create a normal Android UI with a text box or something, and get input from the user that way.
Is it possible to create PDF documents (e.g. on a nightly schedule) with Tableau and have those documents exposed by a URL by the Tableau server?
This sort of approach is common in the Jasper Reports and BIRT world, so I was wondering if the same approach is possible with Tableau?
I couldn't see any documentation on the Tableau site for creating PDFs, other than print to PDF
With Tableau Server, you can access your published workbook in a pdf format with this URL:
http://nameofyourtableauserver/views/NameOfYourWorkbook/NameOfYourView.pdf
Simply, the url is the url of your view + you add ".pdf".
The pdf file will be generated dynamically when accessing the URL.
Another option is to program your own script with tabcmd.
You can have more info on tabcmd here: http://kb.tableausoftware.com/articles/knowledgebase/using-tabcmd
The same technique also works for PNG. You can control filters using ?field_name=value. You can even select multiple values like this ?field_name=value1,value2.
Parameters can be set the same way.
Personally I've had the best luck with discrete dimensions instead of continuous ones.
I use the Windows Task Scheduler with batch files and Tabcmd.
Programs needed:
Tabcmd
Windows TaskScheduler (All Programs- Accessories - system tools)
http://onlinehelp.tableausoftware.com/v8.1/server/en-us/tabcmd_overview.htm
(tabcmd, how it works?)
Batchfile (create a text file and then save with file extension .bat):
1- Locate tabcmd and login
2- use function tabcmd get "http:\..." and -f "C:...pdf" to save to file.
3- concatenate the filters you want to use to the end of your URL as shown in other answers(all filters on the view must be included(filled out))
4- Save Batch file
Windows Task Scheduler:
1- create a task that will execute the batch file
2- TEST
You can do this by typing
http://server/views/WorkbookName/SheetName.pdf?:format=pdf
Another option will be using javascript api like below..
function exportPDF() {
viz.showExportPDFDialog();
}