I have list of 600 companies in an excel file. I want to extract data for each of them in .csv file.
I have started by searching for each company separately and downloading the excel and saving it. It's very time consuming.
Can someone help me with an easier method to solve this.
Related
I am a Bi developer in Microsoft Sql server for a while now.
I have always worked with ether an almost clean data which is excel files with first row is the headings that does have too much rubbish in sheets(like irrelevant data, calculations and so on), text files with data separated by a comma (csv files)
Or with relatively small amounts of files which I cleaned manually(it wasn't an issue)
In my new job I am getting many not clean files, examples are: plain text files (not csv) and excel files the opposite of the mentioned above.
My problem is that these files are many and going through every file is upsetting (opening cleaning manually and trying to make any sense of the data within) so finally I can load it to an integration service tool (ssis, Informatica) and then to a Viz tool through a data warehouse.
Viz tool like Tableau desktop can't clean them appropriately with the automatic interpretation (it takes only the main tables and ignore the others with these not clean files)
I am sure someone worked with these things, your help would be appreciated!
How to deal with these situations?
i'm naive to Big Data field. I started exploring about its tools like Hadoop and got clarity about this framework and Map/Reduce framework but still have a lot of questions:
Actually i want to analyse emails and do some Email categoriation so i can organise emails into different categories but was wondering how should i store those emails into HDFS.
Should i first of all convert my emails to text file (composed of spaced-separated columns: Date, Author, Subject, Content..) or to sequence file composed of binary key-value pairs and then store the file into HDFS ?
I'm not used to work with sequence files but i read many articles about how HDFS store unstructured data into those type of file.
can someone please enlighten me?
Thanks in advance.
Does anyone know how to export a CSV file from a SQLite3 database in an iPhone app?
There are several iOS CSV libraries that can be used to export the data from the phone. This is a trivial data transformation task - you read the information line-by-line from the SQLite result set, and send it out into the CSV file. If you do not need to process the file after the complete read-in, then you should be able to stream the data as quickly as it is read.
Writing CSV files is mostly trivial - and can be implemented by someone without much effort. I'm certain there are libraries for iOS specifically, but I've used the python CSV export routines regularly, and they have been read by excel without much effort. You just have to be careful, as excel has the habit of interpreting the results once they are in there, making some accuracy calculations impossible.
the .csv file format is trivial, and I've implemented it several times in several languages. if you have numeric requirements, though, you will be spending a lot of time making sure that the program that you feed it into is accepting the numbers properly - and you will have to deal with 'I imported it into excel, saved it and the data is wrong now' bugs...
I am producing 5 different Excel worksheets (.csv file) using Perl. I want to combine all these as one file (Excel workbook) with 5 different worksheets rather then have 5 seperate files.
Don't know if this is possible? IF anyone have any idea on how to do this that would be greatly appreciated.
You should use the CPAN Module Spreadsheet::WriteExcel.
It can write different worksheets. At the end there is an example that takes a tab.txt file, and write it to a spreadsheet. If you modify this example to take your five files, and each time create a new worksheet in your spreadsheet, your problem is solved.
I'm guessing this won't apply to 99.99% of anyone that sees this. I've been doing some Sawtooth survey programming at work and I've been needing to create a webpage that shows some aggregate data from the completed surveys. I was just wondering if anyone else has done this using the flat files that Sawtooth generates and how you went about doing it. I only know very basic Perl and the server I use does not have PHP so I'm somewhat at a loss for solutions. Anything you've got would be helpful.
Edit: The problem with offering example files is that it's more complicated. It's not a single file and it occasionally gets moved to a different file with a different format. The complexities added in there are why I ask this question.
Doesn't Sawtooth export into CSV format? There are many Perl parsers for CSV files. Just about every language has a CSV parser or two (or twelve), and MS Excel can open them directly, and they're still plaintext so you can look at them in any text editor.
I know our version of Sawtooth at work (which is admittedly very old) exports Sawtooth data into SPSS format, which can then be exported into various spreadsheet formats including CSV, if all else fails.
If you have a flat (fixed-width field) file, you can easily parse it in Perl using regular expressions or just taking substrings of each line one at a time, assuming you know the width of the fields. Your question is too general to give much better advice, sorry.
Matching the values up from a plaintext file with meta-data (variable names and labels, value labels etc.) is more complicated unless you already have the meta-data in some script-readable format. Making all of that stuff available on a web page is more complicated still. I've done it and it can be a bit of a lengthy project to roll your own. There are packages you can buy, like SDA, which will help you build a website where people can browse and download your survey data and view your codebooks.
Honestly though the easiest thing to do if you're posting statistical data on a website is get the data into SPSS or SAS or another statistics package format and post those files for download directly. Then you don't have to worry about it.