What? DARPA Does Open-Source?

How a machine recognizes a man from a tree seems like a pretty mundane, theoretical question. How a machine differentiates an enemy combatant from a friend or any other object on the battlefield, however, is definitely the sort of question the Defense Advanced Research Projects Agency (DARPA) should be asking. Similarly, making sure a car identifies a pedestrian on the street — under any and all weather conditions — is a significant challenge to the development of advanced driver assistance systems.

Despite recent advances in the embedded vision area, vision algorithms still are an experimental field studded with stubborn problems. Weeding through a growing number of vision algorithms to aid the time-consuming task of generating test objects necessary for better vision algorithms has been DARPA's focus in recent years.

Under a program called Visual Media Reasoning (VMR), DARPA contracted two private companies — SRI International and Next Century Corporation — and completed the development of two general-purpose vision system development tools. One offers the automated evaluation of vision algorithm performance over a massive parameter space. The other enables generation of synthetic image content for use in training and testing detection and recognition algorithms.

Mike Geertsen, DARPA program manager, will present an overview of the VMR program and these enabling tools at the Embedded Vision Summit scheduled for Wednesday, Oct. 2, in Boston.

The most remarkable thing about the latest development is that “these tools will be released as an adjunct to the OpenCV open-source computer vision software library in late 2013 or early 2014,” Jeff Bier, a founder of the Embedded Vision Alliance, told us. “You rarely hear 'DARPA' and 'open-source' in the same sentence.”

Although there are many shades of open-source, Bier said that DARPA's idea is to make these general-purpose tools publicly available in a source code form.

DARPA's open-source move underscores the reality that computer vision is still a developing field, with lots of people trying different ideas and technologies.

If you are in search of the best vision algorithms, your challenge is not only assessing “a whole bunch of different algorithms,” but also looking into each algorithm featured with “10 different knobs” to turn, Bier explained. In other words, “You could end up running through 100 algorithms with 1,000 parameter settings.”

True, computer vision is no longer an academic theory. There has been significant growth in vision algorithms, application developers, and their communities. But that very growth has resulted in “a cluttered landscape of algorithms,” according to Bier.

Under DARPA's VMR program, SRI has developed automated performance characterization tools, providing an efficient means of assessing the performance of different algorithms across imaging domains. The tools identify how well an algorithm will perform for a given image at a chosen parameter setting, and the parameters for a particular algorithm and image.

Meanwhile, Next Century Corp., under the VMR program, has developed tools to generate synthetic images that can be used for the development of vision algorithms.

Why synthetic images?
Noting that 3D synthetic images used in video games these days are very realistic, Bier said that until now, vision algorithm developers had to collect vast numbers of images that required manual annotation. In order to develop vision algorithms, such annotated data has to cover the range of object variations, poses, and environment situations, so that a computer vision system can perform successfully in operational situations.

“By generating these images synthetically — where we know a priori what they are, we can supplement the vast data needed for vision algorithm development.”

DARPA going open-source with these vision tools is certainly a novelty to a lot of people. More importantly, Bier predicts there will be a lot of eager developers lining up to get their hands on them.

This blog was originally posted to EE Times.

11 comments on “What? DARPA Does Open-Source?

  1. Hailey Lynne McKeefry
    September 29, 2013

    This is an interesting instance of collaboration.  The linke to video games developers took me by suprise, but upon reglection seems a natural.

  2. ahdand
    September 30, 2013

    @Hailey: Yes sounds great but I have one worry on Open Source as a common.. What is the reliability scale on it since nobody is there to be responsible for the source code.         

  3. Taimoor Zubar
    September 30, 2013

    “This is an interesting instance of collaboration.  The linke to video games developers took me by suprise, but upon reglection seems a natural”

    @Hailey: I think it makes more sense to hire someone who's really passionate about getting the results such as a gamer than to pay an employee who might not even be fully motivated and committed towards the task. I like what DARPA is doing.

  4. Michael Steinhart
    September 30, 2013

    I wonder whether there's a benchmark that indicates the difference between an algorithm's performance on synthetic images vs. real ones?

  5. Michael Steinhart
    September 30, 2013

    Generally speaking, I think 'opening up' a project is a good way of gathering some free help. I have to wonder, though, when government agencies are concerned, are they giving us the cutting-edge stuff or holding some of the more powerful algorithms back?

  6. Nemos
    September 30, 2013

    This move confirms the forecast that in the future the development model based on openness the “source” will be the prevailing trend. I am quite happy when I am reading such moves as DAPRA – open source that give me the feeling that most of the people in the IT industry understand the important role of the open source and the team playing resulting from this.

  7. Nemos
    September 30, 2013

    It is not about free help, the openness has to do with the way you choose to develop your product. And about security, most of the security algorithms are open source in a way that anybody can see the implementation.  

  8. Hailey Lynne McKeefry
    September 30, 2013

    Open source projects have accountability built in–and it's not just engineers working on these projects. There are also really great quality assurance people (QA) who test and monitor. Many people are using this as a way to demonstrate the good work they do. In some ways, there is more room for innovation adn more personal accountablity than in an closed source project. It's human psychology at its best.

  9. t.alex
    October 2, 2013

    Open source projects attract talents. And anybody can be a tester. Once someone finds a bug they simply reports immediately. And someone else will fix the bug immediately. Fast release cycles make the software much more mature over time. 

  10. Hailey Lynne McKeefry
    October 3, 2013

    My husband is a software engineer and he highly values open source projects. He's worked on them, and he encourages his people to work on them. I asked him about it–and he reports that the end results are often very promising.

  11. t.alex
    October 6, 2013

    Hailey, yes definitely that's true. I have been dealing with lots of opensource software too. With the strong community on the internet, it can get only better and better.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.