This Week I Learned - I

Published on: 2021-08-21

Too small to be updates in their own right, but I gotta tell somebody! Here's what I learned this past week, ending on 8/21/21.

Profiling Python Applications on a Server

In trying to diagnose bottlenecks in a slow running application at work, I learned about different profiling options within Python. For Flask applications, Werkzeug provides a ProfilerMiddlewarethat makes it simple to see which processes are taking the most time. However, because it uses cProfile to do the profiling, it doesn't give you any introspection into multithreaded applications in the non-main thread.

Enabling it is pretty simple. All you need to do is wrap your Flask wsgi instance in the middleware, and any calls you make to the API are automatically returned with a printout of the timing of function calls customized to your liking via the standard library's pstats.Stats interpretation of restrictions.

e.g.,

from flask import Flask

app = Flask(__name__)
...
app.wsgi_app = ProfilerMiddleware(app.wsgi_app, restrictions=[0.05])

Here's where yappi comes in. It's a library that allows you to profile any multithreaded applications at run simply by running yappi.start() and yappi.stop() to indicate when the profiler should in fact, be running. The best thing is, yappi provides a method to convert its statistical output in pstats.Stats() format, the same used by cProfile.

from flask import current_app, request
from flask_restx import Namespace, Resource
import yappi

ns = Namepace("sprocket-rocket", description="delivering sprockets to kewl users")

...
@ns.route("/profile", methods=["POST"])
class SprocketProfile(Resource):
    def post(self) -> dict:
        data = request.get_json()
        yappi.start()
        result = process_sprockets(data)
        yappi.stop()
        response = current_app.response_class(
            response=json.dumps({"result": "upload successful"}),
            status=200,
            mimetype="application/json"
        )
        return response

I chose to hack together an endpoint, and start/stop yappi around the process which I wanted to profile.

Putting Together a Markdown based blog with Gatsby

Due to creative frustrations, my curiosity pulling me in a thousand directions, and just a lack of vision, I've been putting off the creation of my own website for some time. However, finally deploying my Songwriter Graph website I think gave me the encouragement that I needed to get started.

I hadn't ever built anything in javascript outside of a couple of automated frontend testing scripts (the songwriter graph is written in clojurescript, as is the UI I contribute to at work), so I wanted my personal site to be a "gentle" introduction to the language. After a heated "eeny meeny..." session between a few frontend frameworks that specialize in Server Side Rendering (SSR), I decided to roll with making my site in Gatsby (v3 in particular).

The idea of having a personal website / blog, which has pages are written in Markdown has appealed to me since I'd heard of it a few years back. It's surprising just how quickly I was able to get started crafting my own with the combination of GitHub Pages and a couple of Gatsby "plugins", gatsby-transformer-remark and gatsby-source-filesystem.

Within a few hours, I had built a few React components strewn across some Gatsby pages. One component in particular, named blogTemplate gave me all I needed to start rendering Markdown pages as blog posts:

export default function Template({ data }) {
  const { markdownRemark } = data;
  const { frontmatter, html } = markdownRemark;

  return (
    <div className={"blog-post-container container"}>
      <div className={"blog-post hero is-medium has-background-grey-darker"}>
        {/* We render a blank hero-body, and hero-foot with text to
        give the title at bottom look */}
        <div className={"hero-body"}></div>
        <div className={"hero-foot"}>
          <p
            className={
              "title is-size-1 is-inline has-text-white is-family-sans-serif"
            }
          >
            {frontmatter.title}
          </p>
          <p className={"pl-6 subtitle is-inline has-text-white"}>
            {frontmatter.date}
          </p>
        </div>
      </div>

      <div
        className={"blog-post-content mt-6"}
        dangerouslySetInnerHTML={{ __html: html }}
      />
      <footer
        className="footer"
        children={
          <span className="has-text-centered">
            ~ Thanks for reading! If you enjoyed it, consider sharing it with
            someone u like 💜 ~
          </span>
        }
      />
    </div>
  );
}

Here, Template is a template for an individual post. Coupled with a page to list all of the blog entries, I've got a solution for my website's blog that I can continue to improve upon with time.