There are a large and rapidly growing number of remarkable applications of Computer Vision. From autonomous cars to smart manufacturing to augmented reality, inexpensive and ubiquitous cameras have made computer vision tools and techniques more relevant than ever. Cheap cameras and smart algorithms are empowering engineers to find creative solutions to important problems, provide real value to customers, and create products that simply could not have existed before.
Mapillary is a service for crowdsourcing map photos and street view. Using simple tools like smartphones or action cameras anyone can collect photos that are combined into street view. Computer vision on the server-side automatically matches and combines photos across time and across users. Photos are processed with privacy preserving blurring using face detection and license plate detection.
Founded: 2015, Raised: $24.5M
Osmo is an award-winning game system that will change the way your child interacts with the iPad and iPhone by opening them up to hands-on play.
Founded: 2013, Raised: $26.5M
Mashgin is building an object recognition technology that uses 3D reconstruction, computer vision and deep learning to identify items accurately. They are applying this technology to enable faster checkout in retail. Multiple items can be scanned simultaneously and in any orientation. No bar-codes or RFID are required.
Gumgum serves contextual marketing messages in line with content users are actively engaged with, helps marketers and rights holders understand the full media value of their sponsorship investments across broadcast and social media, and analyzes visual content across social media in real time.
Founded: 2007, Raised: $37M
PPT with 40 More Interesting Companies
As part of this course, students work to build a solution to a real computer vision problem as part of a semester-long group project. The technology developed by the companies shown above is intended to show the broad range of possible computer vision applications, and give a little taste for what's happening in the industry right now. Of course, these companies have built some truly remarkable technology that will take longer than a single semester to implement. However, all these companies did start somewhere, likely with a simple demo, or Minimum Viable Product (MVP).
This is the project. Find a real problem, and create a MVP/Simple demo that provides real value for someone.
Projects will be broken into and evaluated in 5 stages:
| Stage | Description | Deliverable | Due Date | Percent of Course Grade | |
|---|---|---|---|---|---|
| 1 | Topic Selection | What CV problem will you go after? | 1 paragraph writeup | 9/14 | 5% |
| 2 | Customer Research | Who are your customers? (Talk to them for extra credit) | 3-5 page writeup summarizing customer research + learning | 9/28 | 7.5% |
| 3 | Literature Review | What academeic literature and existing code/tools will help solve the problem for your customer? How are other organizations solving similar problems? | 3-5 page writeup | 10/15 | 7.5% |
| 4 | Technical Planning | How will you solve this problem? What tools will you use? How will you divide the work across your group? | 2 page planning document | 10/26 | 5% |
| 5 | Final Pitch + MVP Demo | 10 minute presentation + demo of your technology to your peers | Public Github repo with well-documented implementation of your solution, final presentation, and deliverables from previous stages | 12/3 ish | 15% |
There are a huge number of interesting applications of computer vision. Good course project topics will meet a few criteria:
- Use image or video data as primary data source
- Leverage Computer Vision algorithms on top of data
- Provide value to someone
Group + topic assignment details will be discussed in class.
A printed one page writeup with including your name, grad/undergrad status, phone number and email listed, a one-paragraph writeup of your topic proposal, a proposed final technology demo, and your resume. If you would like to collaborate with an industry partner (list on Canvas), include the partner you would like to collaborate with. Given more demand than industry partners, groups will be selected for partnerships based on the quality of proposal. Groups of 3-4 will be made based on the similarity of the proposal and intensity of project.
Here we will generally follow the Lean Startup methodology. A nice summary of the relevant key ideas is Steve Blank's Customer Development Manifesto:
A Startup Is a Temporary Organization Designed to Search for A Repeatable and Scalable Business Model
- There Are No Facts Inside Your Building, So Get Outside
- Pair Customer Development with Agile Development
- Failure is an Integral Part of the Search
- Make Continuous Iterations and Pivots
- No Business Plan Survives First Contact with Customers So Use a Business Model Canvas
- Design Experiments and Test to Validate Your Hypothesis
- Agree on Market Type. It Changes Everything
- Startup Metrics Differ from Those in Existing Companies
- Fast Decision-Making, Cycle Time, Speed and Tempo
- It's All About Passion
- Startup Job Titles Are Very Different from a Large Company's
- Preserve All Cash Until Needed. Then Spend.
- Communicate and Share Learning
- Customer Development Success Begins With Buy-In
A nice intro the these ideas can be found in Steve Blank's The Four Steps to The Epiphany.
Now, this is of course a Computer Vision course, not an Entreprenuership Course. The expectation here is that spending some time conducting customer research will make your engineering work more interesting, relevant, and focused.
Printed 3-5 page writeup summarizing customer research + learning. Strong writeups will provide well-researched answers to these questions: What problem will your Computer Vision solution solve, and for whom? What value will it provide them? What are their pain points? How big is the potential market? Do other similar solutions exist? Would your business have any competitors? Who are they? How are they doing? How are potential customers dealing with these issues now? How much would customer be willing to pay for your product? Are your customers individuals or businesses? Be sure to cite the sources you reference in your research.
There's nothing quite like talking to real potential customers. Earn 0.5 pts on your final grade for each potential customer interview you conduct, for up to 3 interviews. Interviews cannot be with people you knew before starting the class - you have to find your own leads. Interviews may be conducted in person or over the phone. Interviews should focus on the customer's problems and how your particular computer vision solution could help them. Record your interview and include a transcript of your interview questions and your interviewees responses as an appendix to your writeup.
For groups working with Industry Partners (see below), talking with customers is replaced by 3 meetings with industry partners. Working with an industry partner is worth 1.0 extra credit point on your final grade, contingent on 3 successful meetings with your partner.
Part 2 was all about people, Part 3 is all about technology. A thorough literature review can save weeks of wasted time implementing dead-end solutions or re-doing work that others have already done.
A printed 3-5 page writeup citing 10+ sources, and 10-15 slide deck presenting the same information.
Sections should include:
This section should read like the introduction to an academic paper. What academic work is relevant to your project topic? How do these approaches inform yours? What are the strenghts, weaknesses, and history of these techniques?
What open source code and tools are available that are relevant to your topic? How long have they been around? How active are the communities around these tools? What data is available for testing and/or training algorithms? Is labeled data available?
What companies are solving similar problems to yours? It can be tough to tell exactly how proprietary solutions work, but what can you find on the internet? Has anyone reverse engineered these products? If you have access to the product, what can you learn from using it? Are there available talks, documentation, or other resources from their engineering teams?
What will a successful demo look like? What are the 3-6 milestones to a completed demo? How will you break up the work accross your team members? What programming languagues are you planning to develop your solution in? What open source tools will you use (opencv, tensorflow). How will you aquire testing or training data? Do you need training data? Does your data need to labeled or annotated? If so, where will these labels come from? What's your technical plan B if you run into issues?
Printed 2 page planning document.
10 minute presentation + demo of your technology to your peers. Briefly tell the story of the problem you set out to solve, what you learned about the problem in each of the previous stages, and demo the solution that you have developed. In addition to the presentation, final deliverables include a public Github repo containing well-documented implementation of your solution, final presentation, and deliverables from previous stages.
Where possible, group projects will involve working directly with a local business or startup. Industry partners will select computer vision "side problems" or ideas that they find interesting but don't have time to explore, but could be relevant to thier business one day. Ideally, industry partners will meet with student groups 2-3 times throughout the semester to provide feedback + direction. When possible, partners may provide data for students to use.
- Groups may come up with interesting approaches that may be worth pursuing further
- Partners may identify well-qualified potential interns or hires
Guidelines for Industry Partners
- Get work on a real problem
- A little guidance along the way
- Exposure to interesting places to potentially work
- A strong resume booster
For groups working with Industry Partners, the extra credit portion of phase 2 is replaced by 3 meetings with industry partners. Working with an industry partner is worth 1.0 extra credit point on your final grade, contingent on 3 successful meetings with your partner.