The yahoo finance quotes API(http://download.finance.yahoo.com/d/quotes.csv?) provides access to a number of features associated with a company for the current year
(Sector Price
Dividend Yield
Price/Earnings
Earnings/Share
Book Value
52 week low
52 week high
Market Cap
EBITDA
Price/Sales
Price/Book).
The yahoo finance historicaldata api (http://ichart.finance.yahoo.com/table.csv?) provides only the following information for a company for a period specificed (start YYMMDD, end YYMMDD)
> "Open": "15.32",
> "High": "15.35",
> "Low": "15.13",
> "Close": "15.24",
> "Volume": "20126900",
> "Adj_Close": "15.24"
How can I get historical data for Dividend Yield,Price/Earnings, Earnings/Share associated with a company? (because right now I can only get information related to the current year using the finance.quotes API and I cannot get historical data associated with these features)
In order get historical dividends you can add "g=v" to your query as follows
http://ichart.finance.yahoo.com/table.csv?s=MSFT&g=v
This will give you the following fields
Date
Dividends
I would credit where I found this information but I do not remember where I got it. Even as I search other sites I cannot find details on this API at all.
It's going to be difficult to find the data you require for free. This is why people who provide financial data, like Michael Bloomberg, are billionaires.
One option you could try is Quandl. I successfully used their Wiki Stock db as a backup for the Yahoo historical data api, as well as the primary source for some hard-to-find indices. You might be able to calculate some of the data you require from their free databases.
You can look at
https://developer.yahoo.com/yql/console/?q=show%20tables&env=store://datatables.org/alltableswithkeys#h=select+*+from+yahoo.finance.historicaldata+where+symbol+%3D+%22YHOO%22+and+startDate+%3D+%222009-09-11%22+and+endDate+%3D+%222010-03-10%22
For anyone who's interested: heres how to download data from eodhistoricaldata.com:
https://eodhistoricaldata.com/api/eod/AAPL.US?from=2017-01-05&to=2017-02-10&api_token=OeAFFmMliFG5orCUuwAKQ8l4WWFQ67YX&period=d
Call the url with the following parameters:
*eod: 'end of day' this Is only one of many possibilities for download. check their site for the complete list
*AAPL.US: the stock in question
* from and to: the from and to date
*api_token: a token for your account
The result is a csv file, which can be handeled in for instance c# using this example: download csv
Related
I trying doing the manual sanitization, however I am getting a type mismatch error in performing the calculations.
I also need help in sanitizing the data and getting the insight as per the below instructions:
The column sellerproductcount gives you the count of products in the
form '1-16 of over 100,000 results' , and you can parse out the product count 100,000.
sellerratings - this columns gives you the % and count of positive ratings (e.g. 88% positive
in the last 12 months (118 ratings) ) if parsed correctly
sellerdetails - you can use this text to parse out phone numbers, and email IDs of
merchants, where available, so our team can reach out to them.
businessaddress - this will give you the business locations of the sellers. You can parse them
to identify if a seller is registered in the US , Germany (DE), or China (CN).
Hero Product 1 #ratings and Hero Product 2 #ratings - these 2 columns give you the number of
ratings of the 2 'hero products' or bestselling products of this seller.
I have attached the dataset for the same.
https://docs.google.com/spreadsheets/d/1PSqRCnmFgq7v7RzZaCXXoV0Edp_vM7QO/edit?usp=sharing&ouid=115547990006782902200&rtpof=true&sd=true
Most of this type of data prep can be done with string & RegEx functions like REGEX_MATCH(). Here are a few examples based on the data you shared:
Seller Product Count
INT(REGEXP_EXTRACT([Sellerproductcount], '(\d*,?\d*) results'))
1-16 of over 6,000 results >> 6000
Seller Rating (Percentage)
INT(REGEXP_EXTRACT([Sellerratings], '(\d*)% positive'))
92% positive in the last 12 months (181 ratings) >> 92
Seller Rating (Count)
INT(REGEXP_EXTRACT([Sellerratings], '(\d*) (?:total )?ratings'))
92% positive in the last 12 months (181 ratings) >> 181
Business Country Code
RIGHT([Businessaddress],2)
AM Treptower Park28-30Berlin12435DE >> DE
These examples all have very straightforward patterns that are present in all rows so they can be done pretty easily with one simple calculation. However, something like sellerdetails which is unstructured, inconsistent, and sometimes incomplete will be a bit more of a challenge. You will need to use a couple of different calculations and techniques combined together to find what you are looking for, as well as some manual data prep. Here's an example of how you can pull out email but it won't work for everything:
Email
REGEXP_EXTRACT([Sellerdetails], '([a-zA-Z0-9.!#$%&’*+/=?^_`{|}~-]+#[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*)')
Good luck with your data cleaning, I suggest using sites like https://regex101.com/ and https://regexr.com/ to learn more about and help test regular expressions.
my problem is as follows.
I have social media data for multiple networks (facebook, twitter, instagram, etc.) and multiple profiles within each network. I have a date parameter which controls the dashboard based on year/month of the data history to summarize data by month.
I need to record the followers of each network and profile, in aggregate and separately, for each month. This calculation is tricky because I need to aggregate the followers by the latest post of each profile within the month. I would like to view those audience numbers for the month using a filter by network and profile.
I have tried a number of things to isolate followers for one profile and one network, hoping I could expand this to a scenario with multiple, and not been successful. This has included the following calculated fields:
(Given a filter for one network and one profile already applied)
attempt 1:
IF LAST() = 0 then SUM([Instagram Profile Followers]) END // This one pulls in total followers for last month (641.6K)
attempt 2:
LOOKUP(SUM( if MONTH([Date]) = MONTH([Date Parameter]) and YEAR([Date]) = YEAR([Date Parameter]) then [Instagram Profile Followers] END), LAST()) /// Returns total followers in month
attempt 3:
{ INCLUDE DATETRUNC('month', [Date Parameter]): max([Date Parameter])} /// Returns followers first day in month (b/c using max date parameter (which is technically yyyy/mm/01)
I'm not totally convinced a solution exists when I would like to create an aggregate by all networks and profiles together, given the latest post for each would be different days. If I could at least get the latest followers for one individual network and profile filtered, that would be fantastic. Any help greatly appreciated.
I'm looking for a way to generate in an excel or csv file a List for every report on Business Objects server with its user access right.
My report should be like: Folder / Report Name / User or Group Name / Privileges
Is there any way to generate such report?
The query builder has no export function built in – the best I’ve been able to do with it is to copy and paste to Excel, which can be pretty messy.
Here’s an example of a query that should work for you:
select SI_NAME, SI_OWNER, SI_PROCESSINFO, SI_SCHEDULEINFO from CI_INFOOBJECTS where SI_SCHEDULEINFO.SI_STARTTIME>=’2009.05.16.05.00.00′ and SI_SCHEDULEINFO.SI_STARTTIME<‘2009.05.17.05.00.00’ – would be from midnight on the 16th to midnight on the 17th if you?re in the Central time zone during DST ? the results are in local time
I think the QB was intended as an example of what you can do with the BOE SDK and has pretty limited functionality, but it can give you some very useful information without any .net or java development.
Working on a financial application that tracks sales. However, I'm running into problems trying to create a schema for properly tracking the data for reports (the main point of the app).
A purchase is the foundation of the app. It has several associations (listed below). Each purchase is tracked via a year and month field. A year is the smallest unit a user may filter a report by, so I will only have to show data for each month in that year.
# purchase.rb model
class Purchase < ActiveRecord::Base
# Associations:
# belongs_to :partner
# belongs_to :purchase_type
# belongs_to :purchase_category
# Attributes:
# partner_id => association
# purchase_type_id => association
# purchase_category_id => association
# year => year in integer (2013, 2014, etc...)
# month => month in integer ("January" => 1, etc...)
# amount => amount a product sold for in cents ($10.00 => 1000)
# fee => fee for associated partner (if there is one) in cents ($2.00 => 200)
end
The problem is that I need to show an overview for a given year, which breaks things down by how many purchases were completed, what partners completed them, and what were the fee amounts. I solved that by having YearMetric and MonthMetric tables that are updated everytime a purchase is added/updated/removed. So you add a new purchase for a given year/month, and the corresponding YearMetric and MonthMetric rows are found and updated with +/- the appropriate amounts/fees.
This solution works well for the overview page. However, I also need to be able to view purchases in the context of partners, purchase_types, and purchase_categories. If I followed the same strategy as my overview report, I would have to add the following tables:
PartnerYearMetric, PartnerMonthMetric
PurchaseCategoryYearMetric, PurchaseCategoryMonthMetric
PurchaseTypeYearMetric, PurchaseTypeMonthMetric
So everytime I add a purchase, I would be doing up to 8 additional DB updates (8 finds and then 8 updates).
The items I'm reporting on are total purchases made, average purchases (historical comparison), total amounts/fees for the period, top partners by number of purchases and by most fee amounts, etc...
There has to be a better solution than this. "Live calculation" by updating 8 records for every 1 purchase seems a bit overkill.
What you're doing is maintaining materialized views of the data in the application. It's a form of denormalization. That can be OK as an optimization but should not be your first choice. It can be very error prone, especially in the presence of concurrency, and must be done quite carefully.
Instead, when you wish to generate a summary report, use an aggregate to SUM them, COUNT them, etc as appropriate. See aggregate functions in the Pg docs, rails Calculations, rails aggregates.
You may find it convenient to create a VIEW over the query you use, and then access the view from the application.
If you find performance of calculating the aggregates in real time for the summary to be a problem, and you cannot solve it with proper indexing and tuning, then you should think about denormalizing. Rather than maintaining your materialized views in the app, though, consider using triggers in the database; they're much easier to write in a concurrency-safe way.
You may also want to look up PostgreSQL 9.4's enhanced materialized views support.
Background:
Our company uses Sage Line 50 as our accounting software. We generate all of our Sales Orders from this program. We also use MS Access 2010 to store order information and generate our internal shop paperwork and packing slips.
The Problem:
Data entry redundancy. We enter information from Customer Purchase Orders into our Sage software, and then enter the same information into the Access database.
What I am looking for:
A solution that will let me export raw Sales Order data after creating in Sage into MS-Access
What I have tried:
The export process from Sage only allows CSV exports, and is a manual process that cannot be automated at all. I am able to produce this export and then run an append query that will import that data into the Access data, but I am looking to automate this process, perhaps on a daily basis. I have looked into using Crystal Reports (packaged with Sage, which I am told can run automated reports and save in an MS Excel format), but I do not know how to navigate the data files to find the specific sales data I am looking for.
The guys are correct - easiest way to do this would be to use the ODBC driver, however, you must run the correct SQL query and ensure you join the Purchase Order table to the Order item table using the correct fields or it will not return the right data.
One issue you will have is that you only want to pull through the most recent transactions so use a query like this to do that your query would look something like this:-
SELECT *, PURCHASE_LEDGER.ACCOUNT_REF, PURCHASE_LEDGER.E_MAIL FROM PURCHASE_ORDER, PURCHASE_LEDGER INNER JOIN POP_ITEM ON PURCHASE_ORDER.ORDER_NUMBER = POP_ITEM.ORDER_NUMBER WHERE PURCHASE_LEDGER.ACCOUNT_REF = PURCHASE_ORDER.ACCOUNT_REF AND RECORD_MODIFY_DATE >= '2013-04-03 00:00:00'
Alternatively there are many one-click commercial solutions to this problem - do a search on Google for "Sage Integration" and you'll find various suppliers who produce products that will do this for you
What you are trying to do can be automated as I've done it for the last company that I worked for (exporting sales orders and invoices to CSV and to Excel). However, the downer is that it requires the Sage SDK which costs £1500.