In this blog post I’ll show you how to scrape Income Statement, Balance Sheet, and Cash Flow data for companies from Yahoo Finance using Python, LXML, and Pandas.

I’ll use data from Mainfreight NZ (MFT.NZ) as an example, but the code will work for any stock symbol on Yahoo Finance.

The screenshot below shows a Pandas DataFrame with MFT.NZ balance sheet data, which you can expect to get by following the steps in this blog post:

After taking you step by step on how to fetch data from the balance sheet, I’ll show you how to generalise the code to also generate a DataFrame containing data from the Income Statement, and Cash Flow statement.

After creating the Pandas DataFrames, I’ll then show you how to scrape data for multiple symbols, and finally, export everything to an Excel file, so you’ll have output that looks something like this:

This post was last updated in April, 2020.

Prior to October, 2019, Yahoo Finance conveniently had all this data in a regular HTML table, which made extracting the data super easy. Since then, they’ve updated the page with a new structure, which was a wee bit tricker to get the data from. Fortunately, it’s still possible. Read on to find out how.

One of the books that I read over the summer break was “Are we there yet? Insights on How to Lead by Design” by Sam Bucolo.

For the first few years of Sam’s career, he worked as a design consultant. As he gained more design and problem solving experience, he came to realise that he was often getting the wrong brief from his clients.

Sam found that while companies could conceptualise new products or services, the sticking point was aligning this conceptualisation to their business models, which meant that time, and time again, projects significantly fell below expectation or failed completely.

These experiences led Sam to specialise in Design Led Innovation.

My notes:

There are three elements of Design Led Innovation:

1. Customer value: This is the starting point, and is about being clear on who the customer is, and the problem that the business is solving for them.
2. Management Mindset: The correct mindset to create a strategy which has been built around new insights about the customer.
3. Strategic alignment: Ensuring that all business processes and systems are aligned with the customer’s problem.

I’ve been using Hugo for my blog for the last few years, and recently published my first theme - Silhouette Hugo.

I recently received an email with the following question:

What is the simplest way for me to rename the categories Development and Golang into other names, like Projects and Gallery? Is this in partials or template or theme or where?

I figured that this might be a common question, so decided to share it in a blog post.

Here’s how to rename categories in Hugo, in the context of my theme.

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

In this blog post, you will learn how to create a Python app using Flask and the Google API which will:

• Support Google Authentication with Python and Flask
• Restrict access via an OAuth scope, so that the app can only view and manage Google Drive files and folders which were created by the app
• Read and write files on the user’s Google Drive with Python.

By the time you get to the end of this blog post, you’ll have built a basic Google Drive file browser which looks something like:

This blog post is divided up into sections which progressively build up an app which interacts with the user’s Google Drive. If you only want to find out about how to do user Authentication with Google and Python, feel free to stop there.

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

I recently attended a presentation by BNZ’s Chief Economist, Tony Alexander.

One of the topics that really caught my attention was his perspective on the things that influenced the housing market over the decades, from the 1950’s until today.

I wrote up a bunch of notes to summarise his points.

If you’re interested in getting a bit of perspective on the NZ property market, and some of the things that caused it’s near-relentless movement upward over time, then read on.

If you attempt to read encrypted data from a DynamoDB table which has been restored from a backup to a DynamoDB table which doesn’t match the original table name, you may see the following errors:

TL;DR the restored DynamoDB table must have the same name as the original DynamoDB table, and be restored to the same account that it was originally created in.

The recommended way to enable HTTPS in Elastic Beanstalk is to use one of AWS’s load balancers such as the Application Load Balancer (ALB) which supports autoscaling, fault tolerance, and other things.

This blog is about hosting a web app prototype on a single EC2 instance, using HTTPS via Let’s Encrypt, without a load balancer.

Using an AWS ALB costs a minimum of about $18 per month, on top of any other charges you currently have, such as $5 for the t2.micro instance that you may be running your prototype on.

So, if you’ve only got one EC2 instance in Elastic Beanstalk for your prototype, and don’t currently want the benefits of an ALB (fault tolerance, auto-scaling, etc), but do want the benefits of HTTPS (protection from interception, man-in-the-middle (MITM) attacks, etc), read on.

When I first started using CKEditor 5 with create-react-app, I installed CKEditor as an npm module, and imported the ClassicEditor build as recommended by the quickstart.

Development mode (via npm start) worked well, and I was happily integrating CKEditor with React, but as soon as I ran npm run build (which generates the create-react-app production build), I ended up with the following error:

> [email protected] build c:\Dev\scratch\ckeditor-integration
> node scripts/build.js

Creating an optimized production build...
Failed to compile.

Failed to minify the code from this file:

        ./node_modules/@ckeditor/ckeditor5-build-classic/build/ckeditor.js:5:7350

Read more here: http://bit.ly/2tRViJ9

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `node scripts/build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Ruh-roh. What to do next? How can I solve this? Is this solvable? Is there another way?