In this interview recorded 28 March 2011, I talked with Bob Tolbert, the VP of Development at OpenEye about the software engineering processes that they use to develop their cheminformatics and molecular modeling tools.

AD: Hello my name is Andrew Dalke. Welcome to episode three of "Molecular Coding."

AD: Shortly after OpenEye's CUP user group conference in 2011, I dropped by the OpenEye offices to interview Bob Tolbert. He's now the vice president of development at OpenEye. I first met Bob at a Python conference many years ago. He was working for Boehringer Ingelheim, and showed off that their email addresses were so long that they didn't fit on the front of their business cards. Back then he was a OpenEye customer who had written Python wrappers for OELib so he could develop his software in Python instead of C++. That influenced OpenEye to hire Bob and move him out to their headquarters in Santa Fe, where he's been since 2002.

AD: My other two memories of meeting Bob are that he had served in the Navy on a nuclear sub, and that he enjoys talking. That last observation is still true, which made the interview enjoyable, but it also encouraged me to put off typing up the transcript for this podcast for a long time.

AD: This interview took place in Santa Fe, New Mexico on 28 March 2011.

music

AD: Welcome to another edition of Molecular Coding. I'm here with Bob Tolbert at OpenEye. I wanted to talk with him about software engineering in cheminformatics, especially software engineering at a company that develops cheminformatics software.

AD: Can you introduce yourself and talk a bit about what OpenEye does?

BT: I'm Bob Tolbert. I'm the VP of Development at OpenEye. OpenEye writes tools for cheminformatics and molecular modelling. We sell both toolkits and applications. A big part of our business is the fact that we sell toolkits both across cheminformatics and modelling, etc. in C++, Python, Java and C#.

AD: One of the things I wanted to start off with was talking about toolkit development. There's the question I've had talking with other people is, how do you develop APIs? I mean, there's a dozen or so software libraries that exist [in cheminformatics]. Some of them are good and easy to use, some of them are kind of more complicated. How do you go about starting a new API, like when OEChem started or the depiction library?

BT: I think the first thing that drives the API design is the core functionality. Our desire to think about how you don't want things like they were in the past. OEChem is, if you will, like a third generation toolkit. You think of things that maybe even predated Daylight are the first generation, and OELib was a bit of a second generation, trying to use object-oriented [programming] but in a non-abstract way. A lot of the design of OEChem was done to prevent the sins of the past. To avoid things like leaky abstractions.

AD: Do you actually do out and look at existing APIs and how they worked or ...?

BT: What we did with OEChem was look at some existing stuff. You have to realize we started OEChem in 2000/2001. C++ and template support in compilers was relatively new, and there was a decision to do C++ and do it right. Again, some of the things we wanted to avoid were a molecule object which had too many member functions. It was kind of overwhelmingly large and bloated. It used templates not for template's sake but where they made sense for things like predicates and iterators and other things. To remove as much as we could of things like internal implementation from the API. Because if you let those abstractions leak out and people start using back-doors, you're stuck and can't change. This was a fundamental flaw in OELib.

BT: In fact the OEChem API went through I would say a year, year and a half of real argument, real let's-try-again, real throw-it-all-away-and-start-over because it was a clean slate, to some extent, from the API point. What's going to be in a molecule? What's going to be in an atom? Do molecules own atoms or do they just know about them? These kind of things were all core decisions, and in the API we're going to use C++, we're not going to expose STL iterators and things like that. We're going to use our own iterators and make them work the way iterators should work.

BT: Once you build that core then 10 years later [when] you want to design a new library, you can still use that same philosophy to design the next API. If I wanted to design a new API for depict, or a new API for a new library, you can go back to those design principles. There's enough people here now who are bought into those principles that you can get a group of people together and say 'Okay, let's talk about this new design' and it kind of works through argumentation and experimentation. Even some things we've done very recently; we did it all, we wrote it all up, we made it work, we used it for a while, we started writing examples and go "This doesn't feel like OEChem. It doesn't feel right."

AD: Would the depiction library re-write be like that?

BT: That's one of the things we've done. Because the very original API was something that a bit put together out of other pieces, it never quite felt like other things. In the rewrite, what we were trying to do was manage completely replacing the underpinnings so the pictures are better and more flexible and can leverage a lot more stuff that we know how to do now. But not make the API so drastically different that people didn't know what to do. Then when we got it mostly done, we said 'well this is great. We fixed the pretty part. We can make better pictures we can [???] but we've still got this kind of not-great API.' We're expecting people to learn something new, so they're going to learn something new anyway. We might as well bring them all the way and say 'well they're probably an OEChem user - to use this they have to be - so why not push it further down the road of making it feel like it belongs as part of that group?' And further break the tie to the previous API. I think that's something that still a work in progress but I feel like we're getting closer.

AD: When you're doing the API development, you're saying that you build other tools based on the API for internal use, just to try it out -

BT: I think you can't really know if an API is great until you write the documentation and you write the examples and you try to explain how to use it to somebody else. When you do that - before you even actually show it to somebody else, just the act of explaining it to yourself while you're writing that documentation, or writing that example - you find a hole, or a flaw, or a 'wow! I have to do these three things in the right order or it doesn't work; is that a good thing?' Or 'why do I keep cutting and pasting these eight lines into every example? Why is that not a single free function that does those eight things in the right order every time without me having to worry about it.'

AD: That explains some of the evolution of the toolkit I've seen over time, such as the high-level functions for assigning aromaticity appropriate for SMILES. I used to go to the web site and graph those 'eight lines.'

BT: There's a lot of that stuff. Depiction used to have this set of standard things that you had to do. It was boilerplate. You could copy it out of the example. Or you could just write a function that did it. A lot of users wrote that function and called it themselves. We just added it to the API. Now we have some very high level functions that do that.

BT: You know, the important thing is to not only write the high-level functions, because of the stuff that people have. They want to do something different, they don't want to do stero that way, they don't want to depict this or that, they have some special case. The key is to write the low-level open enough that people can be pretty flexible, and then write the high-level when you don't want to have to think, or you want to go and write the simple example.

AD: How do you balance out the needs of the people who want low-level and performance, and the needs of the people who want high-level and not worry about it?

BT: I'm not saying that high-level is not performant. I think that most of the high-level stuff is written to be performant. We don't right high-level stuff that does extra work just because it's easy. We don't really do that.

AD: I can actually give an example of my teaching of the OpenEye code versus, say, the RDKit code and other libraries. OpenEye has the ability to take a molecule and then OEParseSmiles parses a SMILES into the molecule, where as most every other library has "given a SMILES, make a molecule out of it." The OpenEye tools are much faster because they can just reset the molecule rather than parsing into it. I think it's more complicated there for me to teach people that you have to do this two-step process. But I can measure a 10% faster performance just because of doing that.

BT: Yes, I think that's probably true. And there are other things where sometimes to protect people at the first level you have to re-initialize an object rather than re-use it because they don't clear it out. There are a number of things that do that. But I think we tend to write things in reasonably modular level where we decide 'this is a function that other people might need in the company, 'this is a function that only I need because it's an implementation detail and I'm writing this as a function just to break it out', versus 'this is a function that a customer would need to actually be able to do this.' We have three levels. We even have discussions about where different APIs go. Is this going to be a public API, is this going to be an implementation API, is it going to be a private API?

BT: One of the advantages of a private API is not so much that we're hiding stuff from people - it's not hiding performance or anything else - it's that sometimes you don't know, right? Because this is the other example: Let's say that we come up with a new function or a new feature and we're not sure yet whether this is going to be public; it's going to be part of the toolkit. If you make it private, other people in the company, other products in the company, can start to consume it. You could have some kind of internal customer feedback. Then when you're [???], you can say 'oh, well, I'm using it but it doesn't work the way you think it does' or 'it doesn't work the way I need it too', you can refactor it internally first and then push it into the public API. Then you've got a better chance of having the public API that's stable, because people are not going to have to change. They won't be having to give us so much feedback [?incomprehensible?] the way it was, the way it used to be.

BT: Particularly for depict, this is important too, this is such a big API change. We're not changing it, we're actually creating a new API. The old one is still going to be there for an interation or two so that people's code will still build against the old API. The new one is all new classes with all new names. They can learn, they can port, and slowly we'll deprecate the old one.

AD: You mentioned documentation when developing new APIs. How easy is it to tell people [to] document all the code they do, since that's not a very - software developers don't usually like writing documentation.

BT: That's kind of like telling a kid to make their bed and brush their teeth. They know what's good for 'em but it's not always what they want to do. It is a battle. I think that we are getting better, because we've gotten to a size now where most of us are consumers of somebody else's code. There was a time when we were small enough that we all wrote the code and consumed the code ourselves. At that point documentation was only extra work. But the minute you, as an internal customer, say 'I want to use this library that I didn't write' and you have a choice between digging through a header file or opening up a Python wrapper in an editor, or going to the documentation, then you realize 'holey-moley! I can go to the documentation. There's the function, there's an example usage - I can even cut and paste this code as example code and move on.' You do that as a group, I think everyone's had enough of those experiences and they realize they've benefited from somebody else doing documentation, it's a little easier to do it yourself.

AD: If someone from OpenEye is working with the documentation, and finds that it's not well-written enough - they find they don't understand how to use it, for instance - what's the process for updating? Are the ones who discover the problem responsible for saying 'look, here's a thing', publish it in the bug database, or get ahold of the original developer and ask them to -

BT: We track documentation bugs in the bug tracker just like we track toolkit bugs, or any other bug. If there's a thing there ... Sometimes people have a unique position. They may not be the person that wrote the library, but they're writing an example for another app. It shows up something particularly interesting about it, well they can go do it. They can and they will do that, because they have a unique view of this particular thing. It doesn't always necessarily have to be the person that wrote it that understands how to explain it.

BT: I think as a group we've been pretty happy and pretty successful at doing, I think, a decent job. It's always getting better. We're starting to get decent feedback from customers in recent years that notice that we've made it better, and of course any time you get positive feedback, that reinforcement makes it easier to go and write the next amount of documentation. We get to the point where now we have too much, and people are unhappy that it's too much, and you know you can't make people that happy.

I think the other important thing is you have to find a format that is a very low barrier to people doing it. If there's technical problems with writing documentation, they don't want to learn. We used to do it in LaTeX, and we used LaTeX2HTML to generate the HTML, and went to PDF with the PDF version. That was great, if you knew LaTeX. If you didn't know LaTeX, it was bad. If you didn't want to learn LaTeX, it was really bad. I agree too, that you spend all day working on code and dealing with the C++ compiler, to have to then spend your documentation time dealing with the LaTeX compiler; it's pretty annoying, and so I kind of agree with that. But most cases when people want to complain about these things, the easies thing to say is 'okay, well find me a better solution. If it works then you can win; we'll do that instead.'

AD: Are you still using LaTeX or have you switched to something else?

BT: We've switched to the Sphinx stuff. In fact the previous iteration that used LaTeX was based of the then-standard way that Python did their documentation, which I always thought was pretty useful. When we went looking for a replacement for LaTeX, which wasn't just because of LaTeX. The LaTeX2HTML protocol was getting pretty long in the tooth and no one was using it any more and the results were kind of ugly by modern standards. I say 'well, let's go look to see what Python is doing.' This was a couple of years ago and Python had switched to Sphinx, and their documentation looked just that much better. You go look at Sphinx and it's somebody who finally managed to take reStructuredText and make it usable and not yucky and almost fun to do. It's fun because the results are so awesome. They look great! So we went down that path. I think people are still not happy with having to learn a new language, but if you complain about learning reStructuredText then I don't know what else you expect to do except speech recognition -

AD: It should recognize quill, ink, and paper.

BT: Well, people wouldn't want to necessarily probably do that either. I'm not saying it's not a hard problem, but it is worthwhile and we get enough good feedback from people that they recognize the documentation's useful.

AD: What I find interesting about OpenEye is that there's people doing sales and support and there's developers, but you don't have, for instance, someone specializes in doing documentation or someone specializes in doing QA. I think that's pretty unusual for a software company.

BT: It probably is unusual, and it won't last forever. At some point we will have a full-time QA person. I think as it is now that's not a very happy person. And I think at some point QA has to evolve. A single QA person seems like the most hated person around and would not go to lunch with people. It probably would be short-lived, because I think all they do is just make everybody mad by telling them that stuff doesn't work. It's probably a misunderstanding on my part. But we will drive toward that simply because it's impossible at some point for people to test some of the more complicated stuff themselves. It's like proof-reading your own resume - that's just a really bad idea.

AD: You've that the problem that of course all scientific software has is that some of the stuff, how do you test it? If you're developing a new forcefield or new something -

BT: Yes. There are objective tests and there are subjective tests, and "it works the way I think it ought to" versus "the way the world works." Those are apples and oranges, right? If you know what you expect the thing to do then you can write tests and say "this is what I think the answer is." If you're using a force field, you're using an approximation, you are ... for example, one of the things that's new is that we calculate the entropy. It's a [???] rather drawn out process. It's not accurate to the Nth degree like a quantum calculation, it's totally usable; there are known limitations. So then you can't say 'I'll go look up the entropy calculated in some book and put that number in my test and if I get it the test passes, if I don't the test fails.' But what you can do is - once you're finished with the science, once you've published the paper and established the kind of things we ought to get for test molecules - you can turn that into a test to make sure, well, 'if I run it on Linux, and I run it on Windows, and I run it on 32-bit or 64-bit or I call it from Python or Java, do I get the same number?' So I don't have extra errors from floating point roundoff or some other wierd thing going on with the system. At least those kinds of things catch consistency between the platforms, and in 6 months from now, when someone goes and changes one of the moving parts, we suddenly start getting different answers because they changed the optimizer or they changed some other low-lying code. Or it could even be changing something in OEChem that changes the chemistry model that affects the force-field parameters and therefore different answers. Well, you need a test to catch that. You don't want to be surprised that then entropy calculations are different, for a reason that has nothing to do with end of the spectrum.

AD: Have you then developed a bunch of regression tests? Say, 'here are all these validation tests, that when you update a compiler or update the OS you then run through all that stuff?

BT: We have it in C++, the lion share are actually in Python, because that's the consumer API, it's the easiest way to write tests, it's the lowest barrier, again, to people who want to write a test is to be able to go and do it in Python. We also have them replicated in Java, and C# - it's not as many I think of those yet - because you still need to test when we wrapped the C++ into Java does it work the same way as if you wrap it in Python.

BT: At the lifecycle of a software package that's shrink-wrapped and put on a shelf, versus the speed at which we come up with new science and try to put it out. The testing is done in parallel with the documentation, is done with the development, and done in parallel with the science. Every bit of test code you write is code that's not writing the algorithm and so it's a balance. You have to make sure you do both.

BT: I also think most everybody here, anybody that's older than a certain age, that didn't start working out with automated test suites, has to come to the realization that this is a good thing. They come to that because it saves their butt one day. They put in this test, or sombody else puts in this test, and then they run something and all of a sudden the test fails. They go look at the test fail and go 'Oh wow! I didn't even have to debug this problem. I made an error, the test caught it, I go back to my code, and I fix it. I didn't have to spend 5 minutes or 5 hours, or 5 days in the debugger trying to find this wierd crash, because the test picked it up.

AD: I've had the difficulty when I do API development of: if I try to push the tests too early - I've had this argument with people who try to do Test Driven Development where they do the tests first - I don't know what the API is going to look like, so the tests work with API and I change the API and I have a problem that my desire to try to get full, comprehensive tests at the same time I'm doing the API changes, I'm doing the API development, makes it very complicated for me. [Note from the transcriptioniist; That paragraph came our rather poorly! -- AD] So how early do you do the tests?

BT: Wee write examples and tests as all-up C++ programs, so they're not built into a big automated test suite, they don't test every stinking little API point in some unit test way, because that does just [bind you into?] time later. But you certainly can write a functional test early that, you know, if I give it this molecule, what do I get for the entropy? Those first tests are the things you can do from the beginning, and they shouldn't be too much effort to go change if you decide the API needs to [bend?]. The other thing to realize is if you make big complicated object APIs, you're buying into your own trouble. If you keep the objects simple, if you put a lot of your functionality into free functions, then you're first API is really small. That actually works. Even if you did go write every test, you're not writing 100. You're maybe writing 10. You haven't bought the extra work until you need it.

AD: By example of 'heavy' you mean the OELib style where you had 'isMethylHydrogen', 'isOxygen' [and] all these other methods on the atoms, for instance?

BT: Right. There's two problems with that. That's not extensible. If you want to add a new method to that, now you have to go and change the atom API to say it's now this kind of atom. You can decide at some point 'that's silly', so you can do what we do, which is use predicates - free functions that operate on the atom. Those are infinitely extensible; it's easy to add another one. The worst case is when you add a bunch of stuff to the atom API and then you decide it's gotten too big, so now you start having predicates, and now when people want to do something they have to decide 'is this a member function or is this a free function? Where do I look?'

BT: Now, it caused some early confusion to OEChem users. People are used to typing 'mol.' or 'atom.' in an IDE and have it show every method that you could ever do to an atom or a molecule. That's great: until that list is 300 things long and you still can't scroll through it in Eclipse or Visual Studio in any rational way. The flip side is to have no members on the atom and they you have to say 'what methods work on an atom? I have to look through the whole API.' I get that. That's why we don't give people a big list, and that's why a lot of the OEChem manual is broken out in functional areas: "I want to do MCS", "I want to do substruture search", "I want to do reactions", "I want to do whatever" and you go look in those sections and you find the functions which are focused around that, and not just every function that operates on an atom.

AD: On last question about testing. How much coverage tests do you have? How much of your API is covered?

BT: I don't know. In Python, for certainly the older toolkits, it's pretty big. They do touch a remarkably large part of the API. The newer toolkits are obviously further behind and in Java and C# they are further behind as well. But I don't actually know the number.

AD: OpenEye also develops some applications, both command-line tools and graphical applications. I'm curious, when you talk about APIs, as a programmer I go 'I understand how to do that.' When you switch into user interfaces and GUIs, how do you test that? I have no good answer for that. Do you?

BT: I don't think the world has a really good answer for that. I think that's one of the things that's going to drive us to have more full-time QA people; people whose job it is is to figure out how to break it, which you're particularly good at, as a hobby -

AD: Thank you.

BT: - is different than the developer who's going to go through a standard path, because he's trying to make sure something works. It's got to be somebody different than the developer, when you get to GUIs.

BT: We use Qt for GUI development. There is a decent amount of testing frameworks to do that with Qt. By the time when you add that we have the scripting language underneath, we actually can do a lot of testing of our GUI apps by writing Python tests that run inside the interpreter and actually call the same functions that the GUI parts do. You can do some stress testing and some big picture stuff running things inside that Python interpreter. [If] you don't have that, it is a lot harder.

BT: From the command-line, you could go crazy and write stuff that calls every argument with every other argument with random stuff to find things. We don't tend to spend a lot of time chasing those problems. Mostly we write functional tests with commands that we know are reasonable set and have expected input and expected output and make sure that we get the same. One of the things we've spend a lot of time on in the last few months - it's part of the ongoing documentation toolkit effort - is that we're rewritting all the examples; in C++, in Python, in Java, in C#. Simplying, focus them on specific tasks. Then we have tests which run all four of them, with the exact same inputs, and checks that they give the exact same output across the entire suite of things. So not only does C++ continually reproduce the same answers, but if I run the exact same example in Python do I get the same answers I got in C++.

AD: Right, because you have the combinatorial problems of different operating systems, different compilers on those different operating systems, and different languages on top of those. How do you test? You support, what, a dozen plus different architectures?

BT: Actually the world is simplying that for us, but yes. The test have got to be automatable, they gotta be in the tree, they've got something you can type 'make test' while you're doing it.

AD: Do you have all the machines here that you can test them on?

BT: We don't support any architecture on a machine which which we don't have running 24/7.

AD: A real machine or a virtual machine?

BT: We use virtual machines for some of them, simply because it makes good sense to use virtual machines. Our machine room used to look like the shelves at Best Buy. Every one was different, and those were a pain to keep track of. We've switched in some cases to virtual machines for build machine. That actually doesn't work too well when you want to go build build-machines and test-machines for GUI stuff. Some of the newer, more popular, more common Linuxes we have real desktops with 3D video cards that are used for testing GUIs, but the back-end stuff - all the other variations - they're all machines that you can get to.

BT: They're not the kind of thing where you say 'we're going to make this machine dual-boot between, you know, RedHat 5 and SuSE 10'. When it comes time to go do a test, well, one guy wants to test on one platform. We've had that in the past. It just doesn't work. So we want to have every platform available all the time to every developer to go and run the tests on.

AD: Who then does the system adminstration of all those different architectures?

BT: We do. It's not that bad. There's one or two of us that can do most of it. I think system administration is one of these things where you have to say 'what are our choices?' We could hire a full-time sysadmin. It wouldn't be a programmer, it wouldn't be a scientist, it wouldn't be one of "us", and he would hate us and we would hate him or her. It would be the adversarial relationship that exists in most other places where you have a full-time sysadmin trying to baby those machines. Since all of us would hate that - we're not at the point where we're willing to give up control and give up the ability to know what's going on; we're not there yet - so then we have to make this kind of pact and say 'this is the lessser of two evils. Yes, we have to do the work, but yes we have control over what gets done.' I think that by spreading that out - we've spread it out over 4 or 5 people - it's actually worked reasonably well.

AD: Even though you have to have people that are more jack-of-all-trades; doing some documentation, doing some system administration, doing some programming -

BT: If you hire good people you can do that. You can't hire just anybody. But if you hire good people they can do all the pieces. And I think there's an advantage to doing all the pieces because then you appreciate the whole process in a different way.

AD: Most people who come here come from chemistry, so there -

BT: That's not necessarily true! We have a good mixture of folks who have computer science backgrounds as well as chemistry backgrounds. If they come in more as the scientific/science-side developers, then yes, they probably have a PhD, they did some coding in their degree and their postdoc. They have to get up to speed in C++. Maybe they were Fortran guys before. Maybe they were C coders before. They do have a pretty good uphill battle to get up to speed on C++ and up to speed on everything. But that doesn't mean they can't do it and they don't want to do it. They have done quite well. Then we've hired people on the other end. They don't know chemistry but they are GUI programmers or they are graphics programmers; those kinds of folks. They have a computer science background. They don't have a problem picking up the algorithms, they don't have a problem dealing with the tools, they don't say "CVS, how do you spell that?". It's not a problem. The balance of all that has worked really well.

BT: You're not going to go hire somebody who doesn't know anything about cheminformatics or graph theory or anything - works as a quantum mechanics person and wrote Fortran - and come in tomorrow and say "Okay, you're now the lead developer on OEChem responsible for all the wrappers." That's not going to happen. That's silly. But we have a wide spread of stuff that we do. A lot of the stuff is in physics on one end, a lot of the stuff is graph theory and cheminformatics on the other and we tend to put people on the pieces where that's what they do. Then they grow up into the other areas when they've found things they're interested in.

AD: How do you structure the software [development]? Say, cheminformatics. People are doing OEChem and depictions and things like that. Do you have a group that's around that, or is it more fuzzy and some people work 80% on this and 20% of the time on that?

BT: It's pretty fuzzy. We have more products than people so we have multiple products per person instead of multiple people per product. That's just the nature of our business model and us. We don't have a lot of hierarchy when it comes to groups within groups and group leaders. It's mostly a bunch of programmers and most everybody knows what they're responsible for. Everybody knows what part they own. Everybody has the freedom to go digging around in other places if they have a problem. The advantage of everybody seeing everything is that somebody in Boston who's two hours earlier than us can have a problem at 8am Boston time and not have to say 'well, I'll guess I'll have to wait until somebody in Santa Fe gets to work and can go look at this problem in OEChem.' They can get go work. If it's a core algorithmic thing, they're not going to just chuck it in without checking, but they're capable and totally able and empowered to go say 'oh, well here's the problem.' The person here, who's ultimately responsible, will just come to work and find an email in their inbox saying 'here's what I did, here's the crash', or the bug or what doesn't work 'and here's what I think is the fix. Is it okay if I check this in?'

AD: Did this sort of corporate culture start when OpenEye first started or was it something you had to work at?

BT: I think it's been here the whole time. Well, in the very early days it was one product, one person. Each person when they came to OpenEye came to do that thing. Joe Corkery came to do VIDA, Mark McGann came to do FRED, Matt came originally to do OMEGA. Each person managed their own piece. That lasted for a few years but at some point the number of pieces outgrew the number of people, the number of ideas didn't require us to hire somebody new; we have the ideas in here, we needed to go do it. Then we started coming up with 'these are the things we want to do now, and what we want to do in the future; what kind of people do we need to bring in to do it.' But I think the culture of everybody being on the same page and part of the same team is driven mostly a lot by culture. It's also driven a lot by the fact that we manage everything together. We don't have a separate git repository for every little piece, that one person owns and no one else has any write-access to, and you have to go cherry-picking. You have access to everything in one spot.

AD: If everyone manages everything, how do you manage the relationship with the customers. Is anyone free to talk with the customers to get the information they want? Or do you try to put that through one person?

BT: We're pretty open to talking with customers. You have to balance letting people talk to customers only to the extent that it becomes a distraction that prevents them from getting work done. That's usually never the case. I think everybody here knows what they know and what they don't know. I really don't have any fear that somebody is going to go talk to a customer and tell them something that's completely wrong, and set a bad precedent. I think that that's just not a problem. Most people know what they know and what parts of the code they understand, and know who to ask if they get a customer request for something that's different.

AD: How much of the direction of OpenEye is based on direct customer feedback, versus where you all think the science will go for the future?

BT: I think that the science direction is driven a lot by Ant, and his desires and his ideas, and what he sees is the future for the industry and for us. -

AD: So no molecular dynamics -

BT: Well, there's a lot of things. Until they have a proven place in our portfolio, we're not going to do it. But you know, if you'd have asked anybody in this company five years ago would we have a fingerprint toolkit, or all the 2D we have, the answer would have been a resounding "no; are you kidding?". We now have a significant investment in 2D. Not because anything other than the fact that we've now come up with some reasonable ways to put it to work along side 3D; not instead of 3D. Now it's an important player, as part of our toolset, and not just something because somebody said 'can you guys do this?" We have a use for it.

BT: That's the important thing. Because we have a close relationship with a lot of customers, it's not that we go looking for questions. It's not that we say 'we this idea for this new product, what do you think?' It's almost that we have a constant back-and-forth from user-group meetings and other things where we could come up with an idea but have to realize "oh, I talked with so-and-so at last year's CUP about that very same thing." They asked me, I came to the same conclusion, now maybe this is something we should do. That tends to happen an awful lot.

AD: Is CUP the most important way to keep in touch with the users; [to know] what the future is going to be for the users?

BT: I think it is.

BT: We do get decent feedback via support from different things, but some people are funny about that. Some people love writing support and we hear from them regularly. Some people don't want to ask ever, want to figure it out themselves, and suffer in silence. We don't always know. We do spend a lot of time on the road; a lot of time visiting customers. Those are also opportunities for people to say "I had this problem three months ago and I never could make it work." The first question is usually "did you write support"" and the answer usually is "Ahh, I didn't have time for it." We understand that customers are busy. If something impedes their workflow right then, they're really good at working around it. Either using a tool or doing another job or just going to something else on their to-do list. Spending time to writing up a bug report and sending it in is usually not high on their list. I totally understand that, because we're probably all guilty about that in some other product, some person's tool that we use. Nonetheless, I think it is an important avenue for us to hear from people when things don't work. We do occasionally hear from people saying how great something does work, which is nice. CUP gets to be more of those opportunities because the customers can talk. You see a customer give a talk and he uses all your tools and you think "holey-moley - this is why I work here!" Because they went and took a tool that I wrote and turned it into something I never thought about and got some great utility in turn.

One of the things we find, which is an important part of the toolkit perspective, is that - it depends on the company - if you were a turn-the-crank guy, in other words, you buy this piece of software, pour stuff in, turn the crank, results come out, at some point, in some big organizations - this is a corporate culture thing from our customer sites - that boss can say "what value do you bring to this equation? You're just turning the crank. I can hire a cheap person to turn the crank." Toolkits give people the ability to customize things, in a way that a canned-up application doesn't. By customizing things - coming up with a new workflow, coming up with a new algorithm, combining multiple algorithms in a way that no one has really done before - is a way for people to show why they are part of the equation at the customer site. And show why they are added value in the actual workflow in the company. Differently from just a turn-the-crank application. Some people really like that. It's a control thing for some people, it's because they've always been a software developer, or cheminformatics [thing?], but it also is that at the end of the year you can say 'I did this. I built this thing no one else has. It solves particular problems or it found particular leads or it did something that I couldn't have done otherwise.' That doesn't hurt. The customers are happy because they are progressing and advancing in their own company. That's a good thing too.

AD: I'm going to end up with: How did you get in the field? How did you start off working on a nuclear sub, then you came into chemistry, and software development. How did you end up in Santa Fe, New Mexico?

BT: You don't have enough time to get into the whole thing. It's a very weird path, getting out of the Navy and following an old professor who was in Idaho, realizing I was a danger to myself and my fellow man if I worked in a lab, switching to theoretical chemistry, landing in a pharma company doing this stuff because somebody had to do it and growing into cheminformatics and C++. It's a weird thing. I was a customer of OpenEye before I came here. I don't really know to this day how I managed to work this weird path.

AD: I remember you did the Python bindings for OpenBabel -

BT: - OELib -

AD: OELib, right, which became OpenBabel.

BT: I was a very early Python programmer because I was an even earlier Perl programmer. In grad school, and again in my first job, I wrote a Perl program that I couldn't read six month later. That was the last Perl I ever wrote. I switched to Python. At the time there was no OELib there was no nothing else. I had to write my own everything. We don't even think we had Daylight at that time in that company; didn't have access to anything. So I wrote my own little set of tools to do some stuff that needed to get done. After a while I got tired of that and wanted to use somebody else's tool. It kind of grew from there, but I knew I was going to be doing it in Python no matter what else mattered. Really I think the Python stuff is what got me here, it's not what kept me here.

AD: So Python and doing chemistry is what took you all the way to Santa Fe, New Mexico.

BT: Yeah, well, I guess that's true.

AD: And maybe the scenery too?

BT: Yeah. Yeah.

AD: Alright, well thanks for the interview, and thank you all for listening to another edition of Molecular coding.

AD: Cue music.

music

AD: Thank you for listening to Molecular Coding. This podcast and transcript are distributed under the
Creative Commons Attribution-Share-Alike 3.0 Unported license. The theme music was composed and performed by Andreas Steffen. I'm Andrew Dalke.

BT: The only reason I say this is one of the podcasts I listen to, the guy always promises to edit stuff out, and when he says that after you've just heard it, I realize: he didn't actually do it. He didn't even edit out him admitting that "oh yeah, we'll edit it out."