It's definitely the case that once you have a hammer, you begin to see the nails.
A developer I know (a very good one, but without much of a math background) showed me a network routing problem he had. The problem: he was trying to organize a distributed system to process data and pass it to the end user. After spending a week or two doing it by hand, he wrote a python script which played around with parameters automatically (for a fixed topology).
I showed him how to set it up as an LP problem and use glpk to solve it, which imposed no restrictions on topology. He immediately resolved to learn LP. He told me that since then, he used LP to solve several problems he would not otherwise know how to solve (or at least would have solved mainly by fiddling/guess/check).
on the flip side, I work with a bunch of math nerds (machine learning experts and what not) and they often write terrible abstractions and apis. There are a lot of skills that are important in software dev, but I feel we often act as if the math skills are more important than the rest, and I'm not sure that they are in general. Having them is definitely a plus.
Also, math is a very broad field, and a lot of it is specific to certain problem domains. I'm just rambling, not sure if I have a real point ;)
functional only helps when you don't have to deal with things outside of your programs memory space. Functional starts looking the same once you start grabbing stuff from databases, calling rest services, pushing stuff to files, etc
As a math guy I know I'm guilty of looking at a problem and trying to map it to a general math problem, solving that problem, and then trying to map that solution back into code, when really I just should have gone for the brute force three line for loop. When you know enough math you tend to see everything as a potential math problem, and sometimes that can get you into trouble.
Basic mathematical structures are the most useful for programming. Learn about sets, relations, logic, graphs, first class functions, infinite sequences and other basics and you'll be able to solve algorithmic problems much more quickly.
Most programmers think in objects and arrays/lists. While this may result in the most efficient program it's not the most efficient use of your time. Many problems can be modeled much more concisely and easily with structures that are used in math.
Here's a challenge for HN: write an AI for Dawson's chess [1] that can determine whether a position is a winning position or a losing position.
No takers? It's not a lot of work. With a little bit of thinking it's 1 line of Python code, or maybe 10 lines of C/C#/Java because they don't have concise support for math structures.
I think a lot of programmers were turned off to math by boring classes that focused way too much time on how to calculate rather than applying it. This typically happened early in their life and was unfortunately followed by reinforcement in later classes that that is all math is.
The biggest leap I took in math was when I started unlearning this math brokenness due to a great math teacher in middle school. Had this not happened, I doubt I would have been excited about the math.
On my blog, I've been trying to show very cool applications of slightly advanced math such as explaining how the Xbox Live TrueSkill algorithm uses some basic statistics to rank and match people. It's a beautiful application of "statistical machine learning." In addition, I tried to show how the Advanced Encryption Standard (AES) uses some really neat finite field algebra.
The hardest part is overcoming people's fear. This fear has been reinforced so much that you really have to go out of your way to break through the fearful blank stare people get when anything math related is mentioned. I'm just trying to take baby steps now; it's really hard.
Math is simple language that has been refined for centuries. It's sad that so much fear is associated with it. Leslie Lamport's interview on Channel9 goes into this more.
I read your TrueSkill article, and it was very thorough. Coincidentally, I was looking over the papers and things for a few days just prior, trying to figure out all leaps that the authors made. The math was simple, but there are a few gaps to actually implementing the thing! I didn't quite get to the part where I would email them, but I was amazed that you spent 6 months on understanding it. Good for you for being persistent!
As for math, unfortunately it applies to the general public too. It's part fear, but also they keep telling themselves they're not good at it - and not trying in response. Math is a mature field, and as such, there's quite a bit of it to get going. It's tough to catch up years of math without doing it.
A peculiar part of it is that some people correlates math with calculations at an early age, the simple arithmetic operations. Maybe it's unfortunate that it is such an easy metric to see who's better at it - easy negative reinforcement.
One of my algorithms professor in college explained to me that in (research) math, it's all about being creative, seeing things and making connections, and teaching understandings. I try to keep that in mind whenever I read up on math, though I am eager about everything to learn in general!
Although I already knew and understood TrueSkill before your post, it helped solidify my understanding. I hope you write more articles like that in the future :)
Thanks for the kind words! I try to write posts that would have helped me understand a topic. I've found that I can get to a hand-wavy understanding of a topic in a reasonable amount of time and then an actual understanding that I can communicate to others in an unreasonable amount of time :)
Sometimes it takes a month, but the TrueSkill one took about six months.
I've learned that deeply understanding a topic (especially one that has non-trivial math involved) doesn't cause a net loss in the amount of "math fog" in my head, it just moves it to a different place (often one that I never new existed before).
As a schoolboy, it was tempting to think that one day I'd "arrive" in math. I just don't think that's realistic. At best, I've just learned to be comfortable wandering around in the perpetual fog.
I've been out there doing web apps and the like, and I always tried to get better and learn more. Eventually I picked up books with enough math in them to make me fear them, and that was when a sneaking itchy feeling I had about my math incompetence became a full force rash. It was clear it was going to hold me back from the developer I wanted to be.
I went back to school. Now I'm knee-deep in 400 level mathematics and while realizations of utility aren't daily, it is definitely a regular occurance where I see something and think, "ah, so there's a useful way to approach problem x." The other thing is even if some specific technique of MATH 4XX isn't useful, the thought process might be. I've noticed that my approach to problems has been tweaked in a mostly subtle but advantageous way.
Of course it's true. Most of the database fetch print store work wouldn't benefit from this work, but I don't want to do that anymore. Hopefully it proves to be worth the trouble professionally, even though personally it's definitely beneficial.
I came to programming from a mathematics background, and I find it helps me on a regular basis, especially set theory when trying to understand more complex SQL Queries.
Of course, I started with a bachelor's in mathematics and then moved into programming professionally. I am currently working on my masters, so I see things differently from some of my colleagues that got traditional CS degrees or were entirely self taught.
For programmers wishing to learn to do applied math:
I'm very sorry to tell you, the solution is obvious and you've thought of it already: hard graft. Pick a project, work at it, read text books and watch video lectures as you come across unfamiliar territory. Solve lots of problems. Rinse, lather repeat.
(Yes, I've been through that journey, and I can slowly see myself coming out the other side. Took me about 2 years so far, it's still the beginning but it's not scary anymore.)
Tips:
- You ARE smart enough. It's this hard for almost everybody else, too. Our brains were designed to gather fruit.
- Find other people to physically sit and work through problems with. (This is really where college shines).
- Read some meta-math books. Like How to Prove It and How to Solve It.
- Use Gershenfeld's "The Nature of Mathematical Modelling" as a guide. Do all problems. Find other sources on each topic, use those to help you.
- Get on MIT OCW and do 18.085 and 18.086. Do all the problems. If you haven't had linear algebra, calculus, or diff EQs, do those courses too (this will take a while if you have to do all of it from scratch).
- If you want to use (rather than invent) basic stats/ML, all you need is probability, simple linear algebra, simple calculus, and convex optimization (free book from Boyd). For many applications, you need substantially less.
- Spend time in a technical library. Just browse. Find a book that's interesting and in which the first two chapters seem comprehensible (though probably not obvious). Take it home and do all problems.
- Read Peter Szekeres "Modern Mathematical Physics". This book more than any other made me realize math is fascinating rather than scary.
- It sounds stupid, but I learn better (and enjoy it more) when I let myself get genuinely excited about the material. Even the Intermediate Value Theorem is seriously cool when you really think about it.
Remember: there's a lot to learn. You're trying to teach yourself what people spend years paying $50k/year to (barely) learn from lifelong teachers. It's always hard. But it can also be fun, once you're not totally lost.
For me, it wasn't math itself that scared me, it was actually doing calculations by hand. Even though we have calculators and software, we still expect kids to do complicated things like long division and solving equations, and even calculus by hand. (Maybe it's a holdover of the Protestant work ethic.)
If I'd known about things like Gödel's Theorem or mathematical logic as a teenager, I would have seen the big picture.
It is very true that serious users of mathematics (engineers, scientists, even mathematicians themselves) don't really rely on their ability to do calculations by hand; indeed, a lot of them suck at it. Doing it by hand while learning the subject matter gives you a much more visceral experience of what's actually happening, though, even if you use a tool to do it for you most of the time.
Very true. I became a math major in college because it was the only technical discipline where I didn't have to deal with numbers. I calculate the standard deviation the same way everyone else does, I plug the numbers into Excel, but I know what's going on under the hood and I have a better understanding of what the result means. For example, the standard deviation of the sample versus the standard deviation of the mean came up the other day.
There's a well-known quote that says "computer science is no more about computers than astronomy is about telescopes". Often attributed to Dijkstra, but I don't know if there's a reliable source.
The reason kids are made to do this stuff is so that they get real contact with tangible abstractions. Granted, it is not the best way to force this kind of thing down your throat, but from my own school experience they teach a kid "one orange", "two oranges" first, and then simply hammer them with problems. At no point do they explain the concept of numbers as an abstraction, as in "just one" and "just two".
Kids don't get to internalize numbers, and all of what comes after, because it's all built on more complex abstractions from there on. Come to think, they don't even "get" the concept of "abstraction" in itself. The education system as of now (and 200 years ago) still brute-forces through this very basic thing, and many students just don't come through, which is to be expected.
If I didn't know better, I'd swear you read my mind. I've been moderately successful being a developer for the past 12 years without using much math. But, like you, as I've started to delve into search and recommendation features for my CRUD apps, I have come to realize that my math skills stink. Anxiously awaiting your next blog post about how you've solved this quandary. :)
I suppose it also depends why you want to be a "good" developer and a "great" one.
To me, developing software is largely a means to get to a goal. I enjoy it, but only in the same way I enjoy cooking. I personally have no desire to "cook" in a factory for thousands of people. I only enjoy the creativity of creating meals, learning some cooking techniques, and of course eating it. The actual problem of 'how to create the meal' is not interesting in the same way when it scales up to thousands of orders.
In the same way, I personally have no desire to delve into creating search algorithms, complicated physics engines, or complex business logic. I would want to do the creative thinking to move those projects, but not the actual logic and development themselves.
Wanting to do the 'creative thinking' to move projects and ideas is the reason why I now find myself now teaching myself math and advanced math. I have ideas that I want to push through, and I will need people to help me with them, but I'm not going to be able to get the talent I need without being able to 'talk' to them in their language. Also it will help me to see and answer new problems that I am unable to now. This is especially relevant to me as most of the interesting ideas I have are regarding AI and machine learning. I am unable to teach myself from some great books and examples out there because, quite frankly, my math skills are non-existent. It is frustrating because I want to move forward, experiment further with these ideas, but can't do this using pure 'black-box' mentality. I need to understand whats in those black boxes so I can make my own.
how do you separate the creative thinking from the logic? it's like being an architect but not caring how a building stands up. if you don't know whether there's an O(nlogn) solution then how do you know if your (creative) idea will scale?
the idea that technical knowledge and creativity are separate is very restricting. it might seem like that at school when you're surrounded by idiots and the teacher has to cater to the lowest denominator, but to do good creative work in almost any field requires mastering the technical aspects.
I would like to say that I understand Newtonian physics, but have difficulty creating a physics engine. I understand how to render a 3 dimensional environment on to a 2 dimensional screen, but I also have difficulty coding such functionality. Does this prevent me from creating a rich, 3D game? Not at all.
so you build on abstractions / tools created by others? another way of saying the same thing is to ask: why stop at algorithms? why not argue that a programmer needs to understand electronics, or the physics of semiconductors.
i think that's a good point (and have voted you back up from zero :o)
i do think i can make a different argument which makes a better case for maths, but i can also see a valid objection to that. i'll sketch it out anyway:
abstractions / tools cannot make fundamental limitations go away; they can only hide them or present them in alternative ways. so at some point you still have to address limitations in your physics engine, or your renderer. if you refuse to understand the underlying maths then you are (1) very much at the mercy of the api / designer of the tool and (2) at a big disadvantage in looking for workarounds.
the same idea more generally: maths (or logic; the two are closely related) is a very powerful language for dealing with complex systems. if you don't master that language then you are not going to be able to handle the same level of complexity as someone who can.
the objection i see to that argument is that great art doesn't necessarily require breaking technical boundaries (it can even be motivated by constraints).
so yes, i agree - you can make great art by using tools provided by people who understand the technicalities.
i guess my only comeback to that is that the original title is "great developer" not "great artist"...
Regarding cooking: You will never go much beyond a set of frustrating, memorized axioms unless you understand food science and chemistry. I feel as though it works the same way with programming.
You don't need to know the details of a Maillard reaction to know how to saute. This is the difference between a visceral pursuit like cooking and an intellectual one like programming. Most of the people cooking great meals at well-respected restaurants have never read McGee (and probably don't read English very well at all) and would have no problem cooking a better meal than any recent CIA grad.
I disagree. You need to know how food behaves, not why. You need to know how an onion will taste after different processes, not what molecular changes it's gone through. Equally being able to explain the maillard reaction or knowing what Myoglobin is won't help you cook a decent meal.
As a math major, I definitely agree with you. There are a lot of problems that seem to be intimidating or unintuitive but when you try to formalize and generalize them a bit, they may be reduced to problems that have been elegantly solved by great mathematicians. You might still be able to scratch your head for a long time and come up with your solutions but chances are that your own solutions are going to be messy, inefficient and ugly.
I've always felt that if you're a programmer and you're hiding from the so-called "mathy" parts of you're doing, whether it's universal things like algorithms or specific things like machine learning, instead of being given a cute title like "developer" and forming a false dichotomy between that and the idea of a "computer scientist", you should be given the title of "sloppy".
Admittedly, you might be able to get by mindlessly plugging in values into someone else's function without ever having to bat an eye. But when something breaks, you're going to be absolutely screwed.
"Programming is one of the most difficult branches of applied mathematics; the poorer mathematicians had better remain pure mathematicians."
I think developers need to treat math like any other framework - continue practicing. When I was an undergrad and then a graduate student I was always doing math. Now that I work in the business world, not so much (although I do use my statistical analysis skills often). My math skills have degrade some because of it.
As an aside, the picture at the end of the article gave me flashbacks to when I tore my ACL doing backrolls on the wakeboard...ouch!
Statistics and probability (up to an undergraduate level) are necessary if you want to work in machine learning. Category theory is nice (but not necessary) if you want to understand the underpinnings of Haskell.
What I think really makes math generally useful, though, is that, at its root, mathematics is a formalization and generalization of problem solving. That is what formal systems and computability theory, for example, are actually about: how, in general, problems are solved.
What I also find useful is getting practices with bottom up approach. In math the way ideas are developed and presented are first with the low level definitions and axioms, and then from there using logic those ideas are combined to prove more complicated theorems which can be used to solve other problems more easily. Similar to how programs can be built starting off with the low level details and then building abstractions on top of those to make dealing with other problems more easily.
This skill in and of itself is useful regardless of the type of application one develops and how much actual math a problem needs.
im of the mind that programming resides in the logic field, and not mathematics. i am a decent dev with practically no formal math background, and I get around pretty well.
A developer I know (a very good one, but without much of a math background) showed me a network routing problem he had. The problem: he was trying to organize a distributed system to process data and pass it to the end user. After spending a week or two doing it by hand, he wrote a python script which played around with parameters automatically (for a fixed topology).
(See http://www.sce.carleton.ca/faculty/chinneck/po/Chapter10.pdf... an example of the type of problem.)
I showed him how to set it up as an LP problem and use glpk to solve it, which imposed no restrictions on topology. He immediately resolved to learn LP. He told me that since then, he used LP to solve several problems he would not otherwise know how to solve (or at least would have solved mainly by fiddling/guess/check).