Scratch
p/scratch
Explore and collaborate with others
Matt Groh
Deep Angel — AI that erases objects from images
Featured
47

Deep Angel is an artificial intelligence that erases people, animals, vehicles, and more from photographs. Inspired by Paul Klee's Angelus Novus, Deep Angel is an interactive AI designed to both share a glimpse into the future of media manipulation and explore the aesthetics of absence. What happens when anything can be auto-disappeared in images?

Replies
Ryan Hoover
I could use an app that does the opposite of Deep Angel: A tool that adds people to photos. We've all been to a party or gathering where a good friend couldn't join. It would be fun to insert them into a picture as if they were there. Request for product, @le_flaneur. 😉
Matt Groh
@rrhoover Oh just you wait! That doesn't yet exist, but we have something in the pipeline very similar to that concept. As a grad school project at the Media Lab, Deep Angel is a speculative design piece and a great springboard for inspiring new ideas and figuring out something really useful.
lukaszx2
@le_flaneur – Any thoughts of embedding metadata into the exported video or creating a database of erasure, some sort of erasure encoding? I am concerned about the societal consequences of technologies that accelerate the access and ease of altering media artifacts.
Matt Groh
@lukasz_x2 Great question, and I admit I don't know the best answer just yet. Any metadata embedded in media can be altered later on. You could also imagine creating a database of image hashes to check if any image matches a manipulated image. But, of course, an adversary could manipulate the image just enough to avoid a hash collision. You're right to be concerned! I think healthy skepticism of media can be one solution, and one of our goals is to give media consumers a taste of media manipulation such that they're more aware of it when they encounter it in the future.
lukaszx2
@le_flaneur – I agree with your point that a database of metadata could be altered in the future, maybe we found a great use case for blockchain. I also agree with you that we should approach media with a healthy skepticism, but I worry that shortly this technology will be so seamless that we will no longer be able to tell its an altered image with our own eyes. Once we reach this point, we will have to be skeptical of all documentation which will prevent us from having a common narrative. This is why I ask can we also include a mechanism that outputs a flag that a moving image has been altered by erasure.
Matt Groh
@lukasz_x2 We are saving image hashes. The tricky thing with flags is there's AI to erase that too! The issue of media manipulation is not new--it's just at a greater scale than ever. Here's a quote from a 1985 issue of the Whole Earth catalog: "Your Honor, we cannot accept this photograph in evidence. While it purports to show my client in a hotel bedroom with a woman not his wife, there is no way to prove the photograph is real. As we know, the craft of digital retouching has advanced to the point where a "photograph" can represent anything whatever. It could show my client in bed with Your Honor. To be sure, digital retouching is still a somewhat expensive process. A black-and-white photo like this, and the negative its made from, might cost a few thousand dollars to concoct as fiction, but considering my clients social position and the financial stakes of this case, the cost of the technique is irrelevant here. If Your Honor prefers, the defense will state that this photograph is a fake, but that is not necessary. The photograph could be a fake; no one can prove it isn't; therefore it cannot be admitted in evidence. Photography has no place in this or any other courtroom. For that matter, neither does film, videotape, or audiotape, in case the plaintiff plans to introduce in evidence other media susceptible to digital retouching. -Some lawyer, any day now"
lukaszx2
@le_flaneur Thank you for replies. It's good to know that you are saving image hashes, hopefully, there will be a mechanism to display altered hashes. And it's true that the issue of media manipulation is not new, even Roger Fenton’s famous “Valley of Shadow of Death ” photograph of a Crimean battlefield covered in cannonballs was also altered. Stalin's Russia also has a long history of erasing Commissars from official photos. And it's true that now media manipulation is at a greater scale than ever as techniques and tools become more accessible and powerful. Hopefully, as technologists, we also create tools that address these results too. Once again, thanks for the interaction.
Matt Groh
@lukasz_x2 And thank you for your comments and ideas!
Kai Gradert
Very cool. How does this technology differentiate from Adobe Cloak (
)? I do a lot of video editing, and I can't wait for this to become commercially viable. Removing objects from a video is still very time-consuming.
Matt Groh
@kaigradert 100% agree on how time consuming it is currently!! The concept behind Adobe Cloak and Deep Angel is the same. With Adobe Cloak, you manually specify the region that you want to remove whereas Deep Angel does this based on a text input (e.g. remove a person or stop sign or street lamp). Once the object is removed, then Adobe Cloak and Deep Angel are conceptually the same. They are both "inpainting," filling in the image with what makes sense from context. I believe Adobe Cloak uses the Patch Match under the hood whereas Deep Angel uses a deep learning architecture. We're still a bit away from commercial viability, but it's getting closer.
Damjanski
<3
Lewis Bertolucci
Interesting project (the process by which you all created this product)... not sure I'd come back to use it beyond novelty, however. Was hoping it could compete with products like Snapheal.
Matt Groh
@lewis502 thanks for the feedback! Snapheal is very cool, but we're not looking to compete with it. One of the very cool, a bit scary, and extremely relevant aspects of Deep Angel is how automated the photo manipulation is. There are only two user inputs (1) the photo (2) the object name. Essentially, this reveals technology that already exists can very much manipulate media at massive scale.
Kristian Kabuay
This is hella dope! I like that it's not perfect at times and creates some ghostly edits. I'd like to explore this as an art project around the intersections of tech and culture. Here's an output of me doing a live calligraphy piece http://deepangel.media.mit.edu/s...
Matt Groh
@baybayin Thanks!! It's fascinating what happens when the AI doesn't have enough context to re-create a plausible background. We've been calling it "mean collapse" since what's happening is the pixels are collapsing into the average color of the surrounding image. This particularly happens when the AI hasn't been trained on similar backgrounds before. You'll notice darker photos, photos with objects on the edge, and photos with very large objects often get that ghostly edit. Would love to collaborate with you on an art project, and I'd be happy to expose the gif API to you if that's useful.
Emily Hodgins
Hi @le_flaneur thanks for sharing Deep Angel. I love your opening video - something kinda creepy about it! You mention on your site that this project is part art, part tech and part philosophy. I'm interested in the philosophy side - can you tell us a little more?
Matt Groh
Thanks for the question @ems_hodge! The philosophy side is intentionally a bit hidden on the website. If you check out the concept page (http://deepangel.media.mit.edu/a...), you can get an initial inkling of the philosophy behind what we’re doing. And if you’re into mysteries, there’s quite a few Easter eggs and a rabbit hole to discover. Unlike most applications that serve content, we’re providing an opportunity for Internet people to remove the objects that mediate our world and social relations. One might consider this to have a Situationist flare. We want people to explore. In many ways, the Internet has turned inwards in the sense that users more often act upon calculated design choices and algorithmic recommendations rather than an innate curiosity. Here’s an attempt (and admittedly, just a start) to encourage discovery and explore how absence can make us think. In Zen Buddhism, the concept of emptiness represents not void nor nothingness but rather the non-limitations and nondefinition of the infinite. It’s hard to explain, and we’re trying to communicate this concept through the experience of Deep Angel.
Hannah Konitshek
Interesting, love the idea! SnapSeed by Google also did this exact thing.. I don't think Google has continued putting a ton of resources into it though.
Matt Groh
@hannahkonnn Thanks for your interest! Deep Angel and SnapSeed are certainly both applications for editing photos. You'll notice there's a bit of a difference in the kinds of things you can edit (automatically removing objects from images vs. professional photoediting features like brush/crop/rotate/repair)
Ken Cucchia
Props to whoever made this video. Best marketing video I’ve ever seen.
Matt Groh
Thanks @kcucchia!! That was Zivvy Epstein, Max (the dog), and me with a lot of help from our friends!
Giulio Pons
Very interesting. I've tried it but it always answer me "Oopsies, looks like too many people are trying to access the Angel at once. Try again in an hour."
Matt Groh
@ginoplusio Thanks for the support and apologies that you ran into that a couple times. We had about 5,000 users visit the website yesterday, and we weren't totally ready for this scale. The interactive AI is up and running at the moment. Hope you have a chance to try it out!
Giulio Pons
@le_flaneur Hi Matt! I've talked about the project in this article https://www.dailybest.it/tech/de... - I succesfully tried it only once. In the morning (Italian hour) it's impossibile in these days :-)
Yash Kankaria
I'm interested in understanding the ramifications Deep Angel could have in democratizing fake news. Are you guys planning to create something that spots or detects fake images?
Matt Groh
@yash_kankaria Yes! One of the features that we'd like to build is a two-sided market where one side is incentivized to create fakes and the other side is incentivized to detect the fakes. We think this could be fun, informative (as you evaluate fakes, you get much better and learn the idiosyncracies of the AI or any AI), and it could generate data to better detect fakes in the future. If an AI "democratized fake news" to the utmost degree, then eventually, people can no longer blindly trust the media (as they shouldn't today either). Media is all about trust and that's why big media brands succeed. That's especially true in previous eras where news was text and speech based. It's kind of human nature to take photographs as evidence of reality, but they are not necessarily. On of our takes on this tool is that it helps build healthy skepticism for consumers by allowing consumers to become creators and realize how media can be manipulated.
Kelley Muro
I can't wait to take this technology, implant a chip that allows me to censor certain things from my children as they grow up.
María Tatay Sanzsegundo
I'm here to say this is the Black-Mirror-esque app I've ever seen. Congrats to the makers!
Sam Henrichs
I can't say that I've used this prodcut. But as an amateur photoshop user, I can tell you that it's a pain to get objects out of pictures. Although this product doesn't do the best job of this, it does a damn good one for not having a real person on a mouse and keyboard. Well done guys.
Maaz Khan
This is so cool! The applications that are coming out of AI Deep learning is just mind boggling. Use of such products can be endless. Congratulations for a great product. Can't wait to see the complete polished version of this. Cheers!
Matt Groh
@kaayotee Thanks for the support!
Ashot Arzumanyan
Hi folks, I'm getting an error message after I click "Show result". Are you sure it's working?
Matt Groh
@ashotarzumanyan I'm getting the same error message, so I just checked the server. We're getting tons of traffic all submitting images at the same time! I'm looking into a fix.
Matt Groh
@ashotarzumanyan And fixed at least for now! This is certainly our biggest traffic day, and there's a lot we can (and should) optimize to make this run faster and scale to more people
Matt Groh
@ashotarzumanyan Oh, too many people just crashed us again. I re-started the server again and it's ready and working
Kristine Campbell
Love that Deep Angel is inspired by art and spirituality, and that you as technologists at MIT understand that imagining, building and executing this (or any) technology is really about art and the artistic process. Kudos to the Media Lab for keeping it real.
Matt Groh
Thanks @kcinoregon! We had fun throughout the design process, which led us to reading some pretty neat books that we probably wouldn't have read otherwise. Of course, we're still in the midst of figuring out where it should go next, and Product Hunt is another part of that process.
Babken Karapetyan
Good job!
Matt Groh
@babken_karapetyan Thanks for the support!!
Matt Groh
I'm excited to share Deep Angel! Happy to answer any questions
Alexander Smekhov
Far from perfect but ok for a start
Matt Groh
@bitrewards Agreed! It's a start ;)