Pro-tip: Write Python like Python

My last post accused Python of being The Slow of the Internet, not because Python is bad but because bad Python is awful.

In many cases, Python is really not slow for the reasons you think it is

Python is a great glue language, a terrific scripting language, because it provides fantastic facilities for manipulating bulky amounts of data. The terrible language that makes our day-to-day lives slower and more miserable is actually anti-Python.

There are two sides to the Python problem: non-engineers using it to write runtime descriptions of data manipulations performed by non-python backends, and engineers writing it as an expose of their non-python backends.

Between the two groups, nobody is really here for Python.

Python, the slow of the internet.

Unpopular Opinion: CPython is stupidly slow. CPython is the Python you’re using if you don’t know which Python you use.

Before Go, Python had taken a firm hold of the systems admin coding, and huge amounts of Linux tooling is written in Python.

During the Great Python 3 Migration of 2019, Python libraries bloated with people introducing bidirectional compatibility, generally by just grabbing some 3rd-party libraries to minimize the footprint of change.

I’m not going to rant about people not knowing the standard ‘dis‘ module exists, or they don’t know about timeit/%timeit… It’s not really an “optimization” issue tho.

Today’s Linux admin activities are agonizingly slow because so many Python developers hear adages about not optimizing Python code they think that you never need to worry about it, so they have no idea how expensive some very common practices are.

Sadly, CPython makes no-need-for-performance-thinking untrue in one really unfortunate detail, one detail that has been agonizingly inflated by the bloat of compatibility code:

Function call overhead :(

The code from this post is in a Jupyter notebook in my github, here.

If you want to interact with it (run it for yourself), you can either use an online notebook viewer (e.g https://nbviewer.jupyter.org/), or Visual Studio Code has really nice support for notebooks, now.

The golang example is here.

I did a thing

A about a month ago, I dug up an old copy of my MUD language from 1990, and took a crack at refactoring some of it in C++, but I quickly got frustrated with just how much that felt like work rather than fun, especially after my recent foray into golang.

So I have this 29+ year old code, written in original K&R ANSI C, and I want to get it, say, at least compiling.

Why not take a shot at doing it in pure C?

Cold hard cache.

Time to crawl the interwebs. I’m looking for something relatively small and lightweight, a binary blob cache that I can drop into place, import a module in python and have relatively easy access to.

The keys are likely to be large, the blobs may be several MB. I don’t care a great-deal about persistence.

What I’m looking to achieve is something like ‘distcc’ for asset conversion. The backend doesn’t need to know that, it’s just going to get semi-opaque key values that ultimately serve to compartmentalize hash spaces.

CPPCon 2017

I love and hate conventions, so I don’t go to them all that often.

Although I’ve watched CPPCon videos, I hadn’t considered something you attended until this year; I wasn’t really convinced it would be worth going.

The agenda for the first few days proposed some very interesting stuff, and I decided to dip my toe.

Alexa addendum

The other day I gave Amazon a hard time for the difficulty of extending Alexa for your own in-house needs. Well, “skills” are actually more accessible than I thought and there are some tutorials – setup … something … in “under an hour“,  or write a controller with “python with flask-ask“.

That brings it a lot closer to being in the hands of the lay-maker, but the setup on Amazon’s side leaves things in a state that will probably result in much sloppiness. I’m also not clear, yet, whether you are limited to “tell” and “ask” directives or if you can create first-class Alexa commands with it.

Going to get a hub and some lights today, get that working, and then some time this week I’ll see if I can implement “Alexa, open Notepad”.

Challenge to Assistant developers

Apple, Google, Microsoft and Amazon are all competing in the digital assistant field. There are pros and cons to each. Yesterday I received an Amazon Echo Dot (2nd gen) and I’ve been dabbling with it. So far Alexa reminds me more of the command-driven voice apps I tinkered with on the CBM Amiga decades ago than she does of Hal, but there is promising progress.

You’ve heard me rant about Cortana, I don’t have a Siri to tinker with and while I’ve had a highly positive experience with Google’s voice recog, for some reason I just haven’t used it in anger – I’ll try and remedy that in the next week or so.

But having the discrete device like this finally gave form to the notion that’s been percolating in the back of my head this last year or so of watching the assistants emerge: Makers.

One of these assistants needs to cross the bridge for home enthusiasts that empowers them to interact with their devices and applications in their terms.

I should be able to rustle up a few dozen lines of code and create my own little agent that transforms into a feature of my assistant exclusive to me and my ecosystem. Once I can do that, someone will build tools to make it easier and more accessible for idea people who aren’t interested in learning to program.

The problem is that to thrive, it’s going to have to be open. Hold your horses there, I’m not anti-proprietary, I’m just stating this simple fact. Apple and Siri aren’t contenders because the winner is going to be enabling a farmer who got his kids to write a raspberry-pi controller to check on his irrigation system from 10 miles away.

I don’t see Alexa being the winner, because her value in is selling Amazon Prime and Amazon Prime Music subscriptions. I suspect anything beyond that is gravy of the cold, lumpy kind from an Amazon perspective, but I’m happy to be proven wrong.

That leaves Google and Cortana.

Google sells advertising, but they have a history of open APIs. Bringing makers into the fold and enabling personalization has all kinds of value potential to Google, starting with bringing you into the android fold, disinsentivizing you from using Apple or Amazon, etc.

Microsoft is built on the success of open architecture and they’re making a large push to support IoT stuff, so this would be a return to roots for Microsoft that could really make WinIoT attractive, so long as they can manage to focus on the role of a driver and not try to restrict it to “must run on Windows”.

There is one more potential contender, though: Facebook.

Mark has shown an interest in exactly this kind of field with Jarvis. They’ve invested a lot in AI, they like releasing the stuff they build into the community. Their mission statement is “to make the world more open and connected”. This notion fits in with the ideas behind Facebook’s internet.org…

It’s not that it’s currently impossible to do these things, but there’s a huge barrier to entry. For example, the simple task of turning on (or off) my PC? Voice control is largely going to be about inter-action; set the heating in another room, check on the house while you’re away, remind me of something in 30 minutes when I might be in a different room or building or vehicle…

(*Edit: You can actually build your own Alexa skills. But it’s definitely not entry-level stuff)

Pi for kids?

Last year I gave my nephews an arduino starter kit as a present. They’re really bright, so it just made sense to them. Faster than I could demonstrate that wiring up the button and the lights let them create a control they’d already figured it out and wanted to know what else it could do.

 

I realize now it was handing someone of my generation a battery, some cash, and saying “you can power anything with this”. It fuels the path of creativity without nurturing it.

 

This year I was going to give them a raspberry-pi starter kit but it’ll just wind up being a way to run games if there’s not something to fuel and guide their imagination a little.

 

“It can do anything electronic”. “Like what?” “Uh, like, uhm, drive a motor” “Oh, to do what?” “Look, this is why I’m a programmer, just imaginate some stuff, squirt!”

 

And they’re too young for me to point them at “forums”. So:

 

I’m hoping some of you might be able to recommend maybe some kits, books, perhaps online courses or guides for a variety of beginner, entry/kid-level projects that include the programming but perhaps focus on the more physical aspect of doing things. They’re *not* gonna want a series of tutorials on how to make different patterns of blinking lights, they’re going to want something that demonstrates practical potentials and – ideally – gives them reason to go to the store with their parents and see the aisle of potential components they might tap into to fire off their little imaginations.

 

Ideas?

Virtualized desktops vs multi-boot

I’m trying to figure out a good way to compartmentalize/containerize my various Windows desktop usage modes: Development, gaming and general use. This seems like a good way to combat a number of things such as the surface-area of each model (exposure to risk) and things like registry bloat etc.

With fast boot times, multi-boot isn’t a terrible idea. But I kind of have this notion in my head of having a graphical Dom 0 (Win10, Ubuntu, something) where I can see my guests as windows and use them thusly, paste, drag’n’drop between them, etc, but have the option to quickly transition them to full screen at any time.

I found (thanks to Bill Hulley) that KVM can do some of this (https://bufferoverflow.io/gpu-passthrough/) but it sounds brittle/fragile to me, and I’m increasingly wary of this level of hackery for supporting such aggressively-evolving systems as GPUs, stuff gets abandoned, APIs and ABIs change, a maintainer drops out…

Having two user accounts might be an option, but they share the same HKLM and things like MSVC aren’t great at being discrete.

Windows Containers sound interesting, but I don’t know that there’s a way for the app inside a container to expose a GUI on the host, all the instructions I’ve found so far are for server-based containers. So for now, I’m just gonna continue putting all my eggs into the one basket :)

Would the interviewer be right?

Please avoid stating which problem these are solutions for.

During a recent discussion the following code was suggested

int desc(const Node* node, int d)
{
  int l = d, r = d;
  if (node->l) l = desc(node->l, d + 1);
  if (node->r) r = desc(node->r, d + 1);  // edit: thanks, Nico
  return max(l, r);
}
int d2(const Node* root)
{
  return root ? desc(root, 1) : 0;
}

One of the folks in the discussion shook his head and said: “If this was an interviewer, they would want you to do it in one function, though”. Then he presented this:

int d1(const Node* root)
{
  if (root == nullptr) return 0;
  return max(d1(root->l) + 1, d1(root->r) + 1);
}

Then he made the assertion “this is what an interviewer would be looking for”.