Boost.Python and Handling Python Exceptions

11Oct09

The Error has Already Been Set

When calling Python code from C++, one issue you will almost certainly
have to deal with is handling exceptions thrown from the Python
code. Python exceptions are not exceptions in the C++-language
sense. That is, an exception thrown in Python code does not start
stack unwinding in C++ or trigger catch blocks. Rather, a Python
exception is generally indicated by an error return value from a C-API
function call, and information about the exception can be retrieved by
yet more calls to the Python C-API.

Clearly, handling Python exceptions from C++ code requires diligence
and consistent checking of error codes, and, really, who wants to deal
with that? (A: C programmers, apparently.) A more natural system is
one in which Python exceptions are somehow converted to C++ exceptions
at the Python-C++ boundary, and where exception propagation continues
out of Python into C++.

Boost.Python makes it much easier deal with Python exceptions in a
consistent and uniform manner with the
boost::python::error_already_set exception. This C++ exception is
thrown whenever a Boost.Python operation results in a Python exception
being thrown. Consider the following code:

using namespace boost::python;
object obj = ...;
try {
  double length =
    extract<double>(obj.attr("some_attr")());
} catch (const error_already_set& e) {
  // . . . some python exception occurred . . .
}

In this example, the code inside the try block can fail in several
ways that will result in Python exceptions:

  • obj may have no attribute named some_attr, resulting in an
    AttributeError
  • The attribute some_attr may not be callable, resulting in a
    TypeError
  • The return value of some_attr may not be convertible to a double,
    resulting in a TypeError

Each of these failures will result in the Boost.Python system throwing
boost::python::error_already_set. In general, Boost.Python reports
all errors at the Python-to-C++ layer using error_already_set. This
means that it’s much harder to ignore/not notice Python exceptions in
C++. Some people might not like this as much as others, but,
considering the ubiquity of exceptions in Python, it means that using
Python code from C++ requires less mental translation.

Translating to Concrete Exception Types

When using Boost.Python, the error_already_set exception means both
that it’s easier to catch Python exceptions in C++ and that you’re
more likely to do so (since they can’t easily be ignored.) Obviously,
though, this is of limited usefulness if you can’t determine the real
nature of the error. error_already_set is just a signal indicating
that something happened, and it doesn’t tell you what happened
(i.e. the type of the exception.)

In order to figure out the original Python exception, you’ll need to
use the Python C-API. There are three functions that are particularly
useful in this situation:

A simple recipe for translating Python exceptions into C++ works like
this:

  • Catch error_already_set, indicating that a Python exception has
    been thrown
  • Use PyErr_Fetch to get the error indicators, Python objects
    describing the Python exception
  • Use PyErr_GivenExceptionMatches to determine the type of the
    Python exception
  • If the Python exception is not of a type that you want to
    translate, you can keep the exception active with PyErr_Restore
    and allow some other part of your code to handle it.

This is a very straightforward algorithm, and can form the basis for
more complex translation systems. However, it is not without its
complexities.

Specifically, you need to be cognizant of the reference-counting
associated with the PyObjects retrived with PyErr_Fetch. Each of these
references is owned by the caller after the call. That is, their
ref-counts have been pre-incremented for the caller, and it’s the
caller’s responsibility to decrement the counts when done with
them. This seems like a clear case where boost::python::object should
be used, right?

Not so fast. If you immediately wrap the results of PyErr_Fetch with
objects, you’ll run into trouble if you try to use PyErr_Restore,
which takes ownership of the PyObjects you pass it. That is,
PyErr_Restore assumes that you have pre-incremented the ref-counts
on the objects you pass in. See the problem? A boost::python::object
will try to decrement its ref-count when it destructs, but
PyErr_Restore wants the ref-count left alone. The following code
shows the problem:

...
catch (const error_already_set&) {
  PyObject *e, *v, *t;

  // get the error indicators
  PyErr_Fetch(&e, &v, &t);

  // wrap them in objects to
  // ensure ref-count decrementing
  object e_obj(handle<>(e));
  object v_obj(handle<>(v));
  object t_obj(handle<>(t));

  // do some work
  . . .

  // We've determined that we don't
  // want to handle the exception, so
  // we reset it for later processing
  PyErr_Restore(e, v, t);
}

// BOOM!

The problem is that when the objects (e_obj, v_obj, and t_obj)
go out of scope, they decrement their ref-counts, taking them to
zero. However, PyErr_Restore thinks that it owns the refs and
does the same thing, meaning that they get dec-ref’d too many
times, resulting in big problems in the Python garbage collector.

But What About borrowed?

A possible solution to the ref-counting problem above is to use
borrowed references when constructing the objects. A borrowed
reference would not decrement its ref-count on destruction, meaning
that PyErr_Restore would have clean ownership of the the objects
when it was called.

However, this has the downside that ownership of the references needs
to be taken if/when the Python exception is converted into a C++
exception (or otherwise handled), i.e. when PyErr_Restore is not
called.

A Complete Solution

Clearly, then, ownership of the references (that is, responsibility
for dec-ref’ing the objects) should only be taken if PyErr_Restore is
not going to be called. In general, this is the case when you’ve
determined the type of the Python exception and decided to convert it
into a C++ exception. This is where you’ll need to use
PyErr_GivenExceptionMatches.

The following code is a full example of catching error_already_set,
checking its type, converting it into a C++ exception in some cases
(in this case, only if it’s an AttributeError), and passing it on
otherwise:

// Remember AttributeError for later comparison
object attributeError =
  import("exceptions").attr("AttributeError");

. . .

catch (const error_already_set&) {
  PyObject *e, *v, *t;
  PyErr_Fetch(&e, &v, &t);

  // A NULL e means that there is not available Python
  // exception
  if (!e) return;

  // See if the exception was an AttributeError. If so,
  // throw a C++ version of that exception
  if (PyErr_GivenExceptionMatches(attributeError.ptr(), e))
  {
    // We construct objects now since we plan to keep
    // ownership of the references.
    object e_obj(handle<>(allow_null(e));
    object v_obj(handle<>(allow_null(v));
    object t_obj(handle<>(allow_null(t));

    throw AttributeException(e_obj, v_obj, t_obj);
  }

  // We didn't do anything with the Python exception,
  // and we never took ownership of the refs, so it's
  // safe to simply pass them back to PyErr_Restore
  PyErr_Restore(e, v, t);

  // Rethrow the exception (or whatever...this
  // is just an example.)
  throw;
}

It is simple to extend the example above to handle more types of
exceptions, and it’s almost as simple to extend it into a general,
dynamic system for translating Python exceptions.

A Fun Note: import May Stomp on Error Indicators

If you start working with PyErr_Fetch, exception translation, and so
forth, you may find that you’re dynamically importing Python modules
while handling Python exceptions. If so, you could run into an
obscure, undocumented (as far as I know) issue: importing a Python
module can clear the Python error indicators. Consider the following
code:

catch (const error_already_set&) {
  // At this point, let's say that a Python IndexError
  // has been thrown

  // We import Python's IndexError for comparison.
  // IMPORTANT: The import() call may clear the error
  // indicators!!!
  object indexError =
    import("exceptions").attribute("IndexError");

  PyObject *e, *v, *t;
  PyErr_Fetch(&e, &v, &t);

  if (!e)
  {
    // Because the error indicators were cleared, we end up
    // here. This is not what we wanted because...
    return;
  }

  // ...we wanted to logic to take us here.
  if (PyErr_GivenExceptionMatches(indexError.ptr(), e))
  {
    throw IndexError();
  }
}

What this means is that you need to be careful about importing modules
when that may be interfering/interacting with exception handling. The
proper thing to do is to store the error indicators with
PyErr_Fetch, import your modules, and then reset the error
indicators with PyErr_Reset. Then, you can continue with exception
translation (possibly calling PyErr_Fetch again.)


Filed under: c++, programming, python   |  Leave a Comment
Tags: , , ,

How to write boost.python converters

27Sep09

Introduction

The boost.python library is powerful, sometimes subtle, and, when
interacting with python objects, tries to make it possible to write
C++ that "feels" like Python. This is as compared with the Python C
API, where the experience is very far removed from writing python
code.

Part of making C++ feel more like Python is allowing natural
assignment of C++ objects to Python variables. For instance, assigning
an STL string to a python object looks like this:

// Create a C++ string
std::string msg("Hello, Python");

// Assign it to a python object
boost::python::object py_msg = msg;

Likewise (though somewhat less naturally), it is also important to be
able to extract C++ objects from Python objects. Boost.python provides
the extract type for this:

boost::python::object obj = ... ;
std::string msg = boost::python::extract<std::string>(obj);

To allow this kind of natural assignment, boost.python provides a
system for registering converters between the languages. Unfortunately, the boost.python documentation does a pretty poor job of describing how to write them. A bit of searching on the internet will turn up a few links like these:

While these are fine (and, in truth, are the basis for what I know
about the conversion system), they are not as explicit as I would
like.

So, in an effort to clarify the conversion system both for myself and
(hopefully) others, I wrote this little primer. I’ll step through a
full example showing how to write converters for Qt’s QString class. In the end, you should have all the information you need to write and register your own converters.

Converting QString

A boost.python type converter consists of two major parts. The first
part, which is generally the simpler of the two, converts a C++ type
into a Python type. I’ll refer to this as the to-python converter. The
second part converts a Python object into a C++ type. I’ll refer to
this as the from-python converter.

In order to have your converters be used at runtime, the boost.python
framework requires you to register them. The boost.python API provides
separate methods for registering to-python and from-python
converters. Because of this, you are free to provide conversion in
only one direction for a type if you so choose.

Note that, for certain elements of what I’m about to describe, there
is more than one way to do things. For example, in some cases where I
choose to use static member functions, you could also use free
functions. I won’t point these out, but if you wear your C++
thinking-cap you should be able to see what is mandatory and what
isn’t.

to-python Converters

A to-python converter converts a C++ type to a Python object. From an
API perspective, a to-python converter is used any time that you
construct a boost::python::object from another C++ type. For
example:

// Construct object from an int
boost::python::object int_obj(42);

// Construct object from a string
boost::python::object str_obj = std::string("llama");

// Construct object from a user-defined type
Foo foo;
boost::python::object foo_obj(foo);

You implement a to-python converter using a struct with static member
function named convert. convert takes the C++ object to be
converted as its argument, and it returns a PyObject*. A
to-python converter for QStrings looks like this:

/** to-python convert to QStrings */
struct QString_to_python_str
{
    static PyObject* convert(QString const& s)
      {
        return boost::python::incref(
          boost::python::object(
            s.toLatin1().constData()).ptr());
      }
};

The crux what this does is as follows:

  1. Extract the QString’s underlying character data using
    toLatin1().constData()
  2. Construct a boost::python::object with the character data
  3. Retrieve the boost::python::object’s PyObject* with the ptr()
    function
  4. Increment the reference count on the PyObject* and return that
    pointer.

That last step bears a little explanation. Suppose that you didn’t
increment the ref-count on the returned pointer. As soon as the
function returned, the boost::python::object in the function would
destruct, thereby reducing the ref-count to zero. When the PyObject’s
ref-count goes to zero, Python will consider the object dead and it
may be garbage-collected, meaning you would return a deallocated
object from convert().

Once you’ve written the to-python converter for a type, you need to
register it with boost.python’s runtime. You do this with the
aptly-named to_python_converter template:

// register the QString-to-python converter
boost::python::to_python_converter<
  QString,
  QString_to_python_str>();

The first template parameter is the C++ type for which you’re
registering a converter. The second is the converter struct. Notice
that this registration process is done at runtime; you need to call
the registration functions before you try to do any custom type
converting.

from-python Converters

from-python converters are slightly more complex because, beyond simply
providing a function to convert from Python to C++, they also have to
provide a function that determines if a Python type can safely be
converted to the C++ type. Likewise, they often require more knowledge
of the Python C API.

from-python converters are used whenever boost.python’s extract
type is called. For example:

// get an int from a python object
int x = boost::python::extract<int>(int_obj);

// get an STL string from a python object
std::string s = boost::python::extract<std::string>(str_obj);

// get a user-defined type from a python object
Foo foo = boost::python::extract<Foo>(foo_obj);

The recipe I use for creating from-python converters is similar to
to-python converters: create a struct with some static methods and
register those with the boost.python runtime system.

The first method you’ll need to define is used to determine whether an
arbitrary Python object is convertible to the type you want to
extract. If the conversion is OK, this function should return the
PyObject*; otherwise, it should return NULL. So, for QStrings you would
write:

struct QString_from_python_str
{

  . . .

  // Determine if obj_ptr can be converted in a QString
  static void* convertible(PyObject* obj_ptr)
    {
      if (!PyString_Check(obj_ptr)) return 0;
      return obj_ptr;
    }

  . . .

};

This simply says that a PyObject* can be converted to a QString if
it is a Python string.

The second method you’ll need to write does the actual conversion. The primary
trick in this method is that boost.python will provide you with a
chunk of memory into which you must in-place construct your new C++
object. All of the funny "rvalue_from_python" stuff just has to do
with boost.python’s method for providing you with that memory chunk:

struct QString_from_python_str
{

  . . .

  // Convert obj_ptr into a QString
  static void construct(
    PyObject* obj_ptr,
    boost::python::converter::rvalue_from_python_stage1_data* data)
    {
      // Extract the character data from the python string
      const char* value = PyString_AsString(obj_ptr);

      // Verify that obj_ptr is a string (should be ensured by convertible())
      assert(value);

      // Grab pointer to memory into which to construct the new QString
      void* storage = (
        (boost::python::converter::rvalue_from_python_storage<QString>*)
        data)->storage.bytes;

      // in-place construct the new QString using the character data
      // extraced from the python object
      new (storage) QString(value);

      // Stash the memory chunk pointer for later use by boost.python
      data->convertible = storage;
    }

  . . .

};

The final step for from-python converters is, of course, to register
the converter. To do this, you use
boost::python::converter::registry::push_back(). The first
argument is a pointer to the function which tests for convertibility,
the second is a pointer to the conversion function, and the third is a
boost::python::type_id for the C++ type. In this case, we’ll put the
registration into the constructor for the struct we’ve been building
up:

struct QString_from_python_str
{
  QString_from_python_str()
    {
      boost::python::converter::registry::push_back(
        &convertible,
        &construct,
        boost::python::type_id<QString>());
    }

  . . .

};

Now, if you simply construct a single QString_from_python_str
object in your initialization code (just like you how you called
to_python_converter() for the to-python registration), conversion
from Python strings to QString will be enabled.

Taking a reference to the PyObject in convert()

One gotcha to be aware of in your construct() function is that the
PyObject argument is a ‘borrowed’ reference. That is, its reference
count has not already been incremented for you. If you plan to keep a
reference to that object, you must use boost.python’s borrowed
construct. For example:

class MyClass
{
public:
  MyClass(boost::python::object obj) : obj_ (obj) {}

private:
  boost::python::object obj_;
};

struct MyClass_from_python
{
  . . .

  static void construct(
    PyObject* obj_ptr,
    boost::python::converter::rvalue_from_python_stage1_data* data)
    {
      using namespace boost::python;

      void* storage = (
        (converter::rvalue_from_python_storage<MyClass>*)
        data)->storage.bytes;

      // Use borrowed to construct the object so that a reference
      // count will be properly handled.
      handle<> hndl(borrowed(obj_ptr));
      new (storage) MyClass(object(hndl));

      data->convertible = storage;
    }
};

Failing to use borrowed in this situation will generally lead to
memory corruption and/or garbage collection errors in the python
runtime.

For more information on boost.python objects, handles, and reference
counting, see the following:

When converters don’t exist

Finally, a cautionary note. The boost.python type-conversion system
works well, not only at the job of moving objects across the
C++-python languages barrier, but at making code easier to read and
understand. You must always keep in mind, though, this comes at the
cost of very little compile-time checking.

That is, the boost::python::object copy-constructor is templatized and
accepts any type without complaint. This means that your code will
compile just fine even if you’re constructing boost::python::objects
from types that have no registered converter. At runtime, these
constructors will find that they have no converter for the requested
type and this will result in exceptions.

These exceptions (boost::python::error_already_set) will tend to
happen in unexpected places, and you could spend quite a bit of time
trying to figure them out. I say all of this so that maybe, when
you encounter strange exceptions when using boost::python, you’ll
remember to check that your converters are registered first. Hopefully
it’ll save you some time.

Appendix: Full code for QString converter

struct QString_to_python_str
{
    static PyObject* convert(QString const& s)
      {
        return boost::python::incref(
          boost::python::object(
            s.toLatin1().constData()).ptr());
      }
};

struct QString_from_python_str
{
    QString_from_python_str()
    {
      boost::python::converter::registry::push_back(
        &convertible,
        &construct,
        boost::python::type_id<QString>());
    }

    // Determine if obj_ptr can be converted in a QString
    static void* convertible(PyObject* obj_ptr)
    {
      if (!PyString_Check(obj_ptr)) return 0;
      return obj_ptr;
    }

    // Convert obj_ptr into a QString
    static void construct(
    PyObject* obj_ptr,
    boost::python::converter::rvalue_from_python_stage1_data* data)
    {
      // Extract the character data from the python string
      const char* value = PyString_AsString(obj_ptr);

      // Verify that obj_ptr is a string (should be ensured by convertible())
      assert(value);

      // Grab pointer to memory into which to construct the new QString
      void* storage = (
        (boost::python::converter::rvalue_from_python_storage<QString>*)
        data)->storage.bytes;

      // in-place construct the new QString using the character data
      // extraced from the python object
      new (storage) QString(value);

      // Stash the memory chunk pointer for later use by boost.python
      data->convertible = storage;
    }
};

void initializeConverters()
{
  using namespace boost::python;

  // register the to-python converter
  to_python_converter<
    QString,
    QString_to_python_str>();

  // register the from-python converter
  QString_from_python_str();
}

Filed under: Uncategorized   |  7 Comments

New Project: sxs

19Sep09

I started a new project today called sxs (short for “side-by-side”). It’s another translation tool, this one designed to help me read things like newspaper articles. The basic premise is that it takes the source text, splits it into sentences, and then displays each sentence with its translation immediately below it.

Screenshot sxs in action

Screenshot sxs in action

It’s largely functional now, with the glaring exception that it only translates to English. That’s on the hit-list, and it’s largely a matter of UI work.

My “sentence detection” algorithm is also pretty rudimentary, but I think I’ll finesse that as I use the program. It’s deficiencies haven’t really bothered me yet, so it may not get much more love.

You can get sxs here.


Filed under: programming, python   |  Leave a Comment
Tags: , ,

Simplifying branches with virtualenv

31Aug09

In my last post I linked to a review of virtualenv, a tool for creating “isolated Python environments”. In a nutshell, virtualenv builds sandbox installations of Python to which you can “switch” in order to isolate yourself from other installations on your machine. Each sandbox has its own Python executables, libraries, and headers, so, for example, packages installed in a sandbox won’t interfere with your primary system installation (or other sandboxes.) This has many uses, and I’ve been using it to help manage code branches for Python projects.

Sometimes the easiest way to work with Python code is to simply install it, i.e. to site-packages. By doing that, it’s trivially available to Python, doesn’t require any changes to PYTHONPATH, etc. It also has the added benefit that you’re generally dealing with your code in a more “natural” state, so you avoid many potential configuration issues.

Of course, if you’re working with multiple branches, you can really confuse yourself if you continually re-install between them. Unless you’re really diligent, you can end up with mismatches between libraries, scripts, unit tests, etc. The problem can be compounded by needing to keep track of different versions of libraries you depend on. Plus, it’s a real pain to be constantly having to check yourself and your environment.

This is where virtualenv can really save you a lot of trouble. By creating a virtualenv sandbox for each branch, you make all of the above problems go away for (almost) free. You get independent installations of each branch’s code, their associated libraries, scripts, and so forth, which will never get polluted with code from the other branches.

The formula I follow is pretty simple. Suppose I have a directory structure like this:

project_name/
  branch_name/
    src/
    unittests/
    resources/

To create a virtualenv sandbox, I would do something like this:
cd project_name/branch_name
virtualenv python
This will create a new directory structure like this:

project_name/
  branch_name/
    src/
    unittests/
    resources/
    python/
      bin/
        {activate, easy_install, python, etc.}
      include/
        pythonX.Y/
      lib/
        pythonX.Y/
          site-packages/

That new structure should be familiar to any Python developer; virtualenv has created a whole new python installation for you, copied from your main installation (or, whichever Python executable was used to run virtualenv.)

Notice the file python/bin/activate. This file contains the code needed to bootstrap you into that sandbox. To use it, do something like this:

cd project_name/branch_name/python/bin
source activate

When you do this, activate does two main things:

  • Update your PATH to see the sandbox’s bin directory
  • Update your prompt to indicate the sandbox directory

By updating your PATH, this script effectively makes this sandbox installation your active Python installation. Installed libraries will go there, it will be the source of site-packages, etc. This is the primary magic of virtualenv: by giving you a simple way to create and activate sandbox environments, it gives you a convenient way to manage multiple, possibly conflicting code bases in a very natural way.

The prompt updating is a good idea, but it could use some polishing. What it does is prepend to your prompt the name of the directory which you supplied to the virtualenv command. So, if your original prompt was

user@host%

based on the scheme I’m describing here it would get changed to

(python)user@host%

While this at least lets you know that you’re in a sandbox, because of how I structure things (i.e. by putting all of my sandboxes in a directory called ‘python’) all of my sandbox prompts look the same. I would rather that they actually tell me the name of the sandbox. You can do this by editing activate. Look for a line near the end of the file like this:
PS1="(`basename \"$VIRTUAL_ENV\"`)$PS1"
and change it to this:
PS1="(BRANCH_NAME)$PS1"
Now, when you source the bootstrap file, you’ll get a prompt like this:
(BRANCH_NAME)user@host%

There’s more to virtualenv than I’ve covered here, so it might be worth checking out if what I’ve described here tickles your fancy. virtualenv is a good idea, well executed, and hopefully you might find it as useful as I have.


Filed under: programming, python   |  Leave a Comment

Doug Hellmann on IPython and virtualenv

18Aug09

Here’s Doug Hellmann’s overview two excellent python tools, IPython and virtualenv. IPython is an enhanced python interactive shell. virtualenv is a tool for creating “sandbox” python installations.

http://www.doughellmann.com/articles/CompletelyDifferent-2008-02-ipython-and-virtualenv/index.html


Filed under: programming   |  Leave a Comment
Tags: ,

Visualizing in-place sorting algorithms

10Aug09

This guy’s work on statically visualizing sorting algorithms is beautiful and awesome.

http://www.hatfulofhollow.com/posts/code/visualisingsorting/index.html

http://www.hatfulofhollow.com/posts/code/timsort/index.html


Filed under: programming   |  Leave a Comment
Tags:

Morale-oriented programming

27Jul09

I want a new language. Heck, it doesn’t even have to be a new one; just rename an old one. But I want this language to do something that no other language has tried to do, something easy but too-long overlooked: help a programmer fell good about him/herself.

Expressive syntax? High-level abstraction? Just-in-time compilation? While great, these things don’t make you feel awesome about yourself. What can? It’s simple: a language named ‘bad-ass’.

Imagine it. You’ll spend all day working in a bad-ass editor, writing bad-ass code. You’ll develop bad-ass modules and libraries, and maybe link them into big, bad-ass applications or bad-ass websites. You may occasionally have some bugs, of course, but your bad-ass debugger will make short work of that. If some marketing suit ever looks over your shoulder and asks what you’re doing, you can say “It’s bad-ass.” So true.

And at the end of the day, as you finish your last cup of coffee and contemplate what you’ve done, you’ll be able to hold your head high because there will be one thing you know for certain: You, sir or madam, are a bad-ass programmer.

I bet you’re already feeling better! And you haven’t even written a single line of bad-ass code in your life. Now imagine feeling this good every day. You can, because the future is bad-ass.


Filed under: programming   |  Leave a Comment
Tags:

Review: The Elegant Universe

25Jul09

I recently finished reading Brian Greene’s The Elegant Universe: Superstrings, Hidden Dimensions, and the Quest for the Ultimate Theory, and I enjoyed it so much that I thought I should write a bit of a recommendation for it. As you probably gather from the title, it’s essentially a book about the state of modern string theory: what it says, where it’s from, where it’s going, etc. However, the book is aimed squarely at curious lay-readers, and in the course of getting to string theory it covers both relativity and quantum theory in ways that most people should be able to follow.

And it is the author’s ability to cast complex, abstract ideas in everyday experience that is probably the book’s greatest strength. Greene doesn’t shy away from talking about spacetime or hidden Calabi-Yau dimensions, but he always finds ways to make them…well, approachable. I came away from the book with a good understanding of the spirit of the topics (or at least, I think I did!), but I never felt forced into really uncomfortable territory; I got the best part of the sausage, but I didn’t have to see it made. If, as Einstein said, “You do not really understand something unless you can explain it to your grandmother”, Greene seems to have proven that he knows what he’s talking about.

In some ways, Greene uses Jared Diamond’s technique of, for lack of a better term, “appropriate repetition”. He manages to repeat important points at the right times and in the right contexts so that, even if you don’t fully grasp them at first, they become clearer and clearer each time. I imagine that this is a difficult technique for a writer (the “appropriate part, not the repetition), but it’s brilliant when it’s done correctly.

On the downside, I could have done with a little less discussion of the author’s own work. While I’m sure it was important to the development of string theory, he tends to get into details of his work that feel out of place. The book drastically changes pace when he gets into the minutiae of his contributions, and it breaks up the flow of ideas at some awkward times.

Also, I’d like to make a comment about the format. I read this book on a Sony eReader and, for whatever reason, all of the diagrams were missing from the text. Considering how much he relies on pictures to communicate complex ideas, I think I really missed out on some of the best parts of the book.

I don’t want to end on a down-note, though! All in all, this is a brilliant book, and I recommend it to any fan of popular science or anyone who just wants to know what nerds are doing with particle accelerators these days.


Filed under: review, science   |  1 Comment
Tags: ,

Euler 79 is in the can

25Jul09

Wrote up euler 79 today. From what I see in the forums, most folks just did this one with pencil and paper. Given the nature of the problem, this makes good sense, but I felt obliged to program. In the end, my code was another brute force of the search space, so nothing too interesting there.

It is worth noting that this was my first submission (and really, my first non-trivial program) in D. D’s philosophy of taking the best parts of C++, jettisoning the bad, and adding its own set of goodness is something that really grabbed my attention. It’s also apparently catching the eyes of people like Andrei Alexandrescu, so maybe there’s something to it.

It didn’t really have to push on the language very hard for this problem, but I’m hoping that the euler problems will give me a good way to feel my way through it. I was impressed by how natural it felt. With my background in C++ and python, I was able to guess at how many things worked an I was right a good amount of the time.

With that said, the documentation for D is pretty shoddy right now. The documentation exists, but I find it hard to find what I’m looking for. Perhaps I should buy the book or shut up and do something about it.


Filed under: euler, programming   |  2 Comments

Euler 78

19Jul09

Finished Euler 78. My solution is here.

The problem dealt with partitioning: how many distinguishable ways can you make collections of indistinguishable objects. My initial thoughts on the solution revolved around some notion of permutations of partitions, and it was totally wrong. As it turns out, there are a number of recursive definitions that allow you to calculate partition counts.

I picked one that seemed tractable and easily programmable, and it worked. Using my initial implementation, however, yielded the result only after an hour or two of computation; this is always a sign that you’ve screwed something up on an Euler problem.

A little examination showed that I was doing a lot of unnecessary recursive computation that could be short circuited. I would have felt smarter if I had noticed this before I submitted my answer, but, in any event, I got the answer right on my first submission. Adding the short-circuiting brought the runtime down to around a minute…much better.


Filed under: Uncategorized   |  Leave a Comment
« Previous Entries