Python

How to scrape stock upgrades and downgrades from Yahoo Finance

In this blog post I’ll show you how to scrape stock upgrade and downgrade data from Yahoo Finance using Python, Beautiful Soup, json, and a bit of regular expression magic.

This technique can be applied to any stock symbol on Yahoo Finance, but for this blog post we’ll be scraping data for Apple (AAPL).

I’ll be guiding you through the process, describing my thought processes and techiques, and if you follow along, by the end of this blog post, you’ll have extracted stock upgrade and downgrade data in a Pandas dataframe which will look something like this:

From there, you can easily export the data into an Excel file, or if you’re a more advanced Python user, host the code in a Web API which can be accessed by Google Sheets, or your own custom code.

Continue Reading...

How to scrape Yahoo Finance and extract fundamental stock market data using Python, LXML, and Pandas

In this blog post I’ll show you how to scrape Income Statement, Balance Sheet, and Cash Flow data for companies from Yahoo Finance using Python, LXML, and Pandas.

I’ll use data from Mainfreight NZ (MFT.NZ) as an example, but the code will work for any stock symbol on Yahoo Finance.

The screenshot below shows a Pandas DataFrame with MFT.NZ balance sheet data, which you can expect to get by following the steps in this blog post:

After taking you step by step on how to fetch data from the balance sheet, I’ll show you how to generalise the code to also generate a DataFrame containing data from the Income Statement, and Cash Flow statement.

After creating the Pandas DataFrames, I’ll then show you how to scrape data for multiple symbols, and finally, export everything to an Excel file, so you’ll have output that looks something like this:

This post was last updated in April, 2020.

Prior to October, 2019, Yahoo Finance conveniently had all this data in a regular HTML table, which made extracting the data super easy. Since then, they’ve updated the page with a new structure, which was a wee bit tricker to get the data from. Fortunately, it’s still possible. Read on to find out how.

Continue Reading...

Google Authentication with Python and Flask

In this blog post, you will learn how to create a Python app using Flask and the Google API which will:

  • Support Google Authentication with Python and Flask
  • Restrict access via an OAuth scope, so that the app can only view and manage Google Drive files and folders which were created by the app
  • Read and write files on the user’s Google Drive with Python.

By the time you get to the end of this blog post, you’ll have built a basic Google Drive file browser which looks something like:

OAuth Google Drive Scope

This blog post is divided up into sections which progressively build up an app which interacts with the user’s Google Drive. If you only want to find out about how to do user Authentication with Google and Python, feel free to stop there.

Continue Reading...