How to produce a JSON tree with nested data from Scrapy

This was an interesting puzzle: creating one single well formed JSON from a hierarchy of web pages. E.g., the sporting goods hierarchy of an e-commerce site could be Categories, Brands, Products. And so you'd like to output JSON like this: https://gist.github.com/dogweather/cb364044df65bd104a0b051f257ae1f6 [Etc.] As an aside, I like architecting my systems to generate this kind of … Continue reading How to produce a JSON tree with nested data from Scrapy →

Making mistakes: Django startproject and Rails new

I like to see how software reacts when I step off the happy path and make a mistake. Today I found this interesting difference with an unknown (or misspelled) command line option: "--derp": https://gist.github.com/dogweather/aac2dce97936484e765c04105ef6fa06 https://gist.github.com/dogweather/937420a48708ee36f3d0268161d61b80 An interesting difference! Personally, Django is reacting like I'd expect. And so for me it's following the principle of least … Continue reading Making mistakes: Django startproject and Rails new →

The perfect IDE in pictures: Part 1, it works for you

I don't think the perfect IDE exists. But all the right ingredients are out there. Allan MacGregor wrote a post about VIM as the "perfect IDE" that started an interesting discussion on dev.to. But I'm skeptical about the pages-long config file, the dozens of independent plugins, and whether the result is really an "IDE". This reminded … Continue reading The perfect IDE in pictures: Part 1, *it* works for *you* →

Testing the multi-subdomain Rails app

Writing tests for a multi-tenant, multi-subdomain app turns out to be very tricky to figure out, e.g.: I saw that, and understood the frustration. Integration tests ("request specs" or "feature specs") are built on a stack of frequently changing libraries and shifting API's. And the recipe for subdomain-aware testing isn't documented in any particular tool's notes. … Continue reading Testing the multi-subdomain Rails app →

What is a Design Pattern? (Answered.)

I recently answered that question: The entire concept of Design Patterns for software was popularized and maybe invented by the Gang of Four. This is the book. It's an amazing artifact, organized differently from any other book I've seen. It's worth checking it out from a library and reading at least the introduction and a … Continue reading What is a Design Pattern? (Answered.) →

Twitter scam: Ethereum / BasicAttentionToken “giveaway”

E-coin faucets have been around for a while, and I’ve never had a problem with them. So I clicked the link, and saw the dwindling amount left to give away.

I switched from iPhone to the Pixel 2: One-week report

...the bottom line is, the experience is simultaneously high-end luxury yet while staying informed and in control of the device.

The Economics of Programming: Externalized vs. Internalized Costs

Many days I feel like my work as an agile consultant is simply internalizing (externalized) costs. First example that comes to mind: software development done too quickly which creates technical debt as it goes. In the short term, a project like that can seem very successful, exceeding expectations for delivery time and customer satisfaction. And then … Continue reading The Economics of Programming: Externalized vs. Internalized Costs →

Python vs. Haskell round 2: Making Me a Better Programmer

Another point for Haskell In Round 1 I described the task: find the number of "Titles" in an HTML file. I started with the Python implementation, and wrote this test: Very simple: I had already downloaded the web page and so this function had to do just two things: (1) read in the file, and then (2) parse … Continue reading Python vs. Haskell round 2: Making Me a Better Programmer →

Python vs. Haskell round 1: Test Output

The point goes to Haskell On the surface, it may sound silly to compare these two languages because they're about opposite as you could get: Python is interpreted, dynamically typed, and slightly weakly typed as well. Haskell on the other hand, is compiled, statically and strongly typed. But they're both open source, and they both … Continue reading Python vs. Haskell round 1: Test Output →