Danna Gurari

Understanding the Broader Picture: The Image and Video Computing Group

Anonymous — Tue, 11 Jun 2024 17:53:35 +0000

Understanding the Broader Picture: The Image and Video Computing Group Anonymous (not verified) Tue, 06/11/2024 - 11:53

Grace Wilson

Assistant Professor Danna Gurari's research revolves around using computers to analyze images and videos. Much of her recent work focuses on improving digital privacy for people who are blind or have low vision (BLV) as well as on analyzing medical images.

When asked what motivates her research, Gurari said she's driven by the desire to make an impact on people's lives.

"I try to find problems based on real-world data in real-world applications. That kind of data is difficult to get, but that's what really excites me. That and mentoring students," Gurari said.

Let's learn more about the research her lab members are pursuing:

Atharva Peshkar, second-year PhD student

Atharva Peshkar is improving radiation treatment for cancer patients by creating a personalized avatar that models each patient's body and breathing patterns.

The small movements of a patient's body are used to minutely adjust the position of the radiation gun in real time. This ensures the radiation stream moves with the body and stays directed on the tumor, as opposed to targeting surrounding tissue.

This innovation was recently recognized by the American Association of Physicists in Medicine as a "Best in Physics" abstract, an honor received by less than 1 percent of submissions.

When asked why he chose Gurari as his advisor, Peshkar said he talked to lab members before joining.

"They said she was really supportive and I thought I should go to her lab because a PhD is going to take five years, and I need an advisor who understands my goals," Peshkar said.

He said Gurari has been a great guide, and so have his group mates, like Myers-Dean, who helped him with CU's computing infrastructure when he first arrived.

Everley Tseng, third-year PhD student

Everley Tseng's research largely focuses on supporting visual privacy preservation for people with low vision and blind people using AI.

One project she contributed to involves a contrived dataset of private content. The team, which has researchers from the �鶹ӰԺ, University of Illinois and the University of Washington, created fake "private" items, like pregnancy tests, prescription pill containers and bills that BLV people took photos of.

This dataset is useful for research aimed at preserving blind and low-vision people's right to privacy.

"It's important for me to pursue this research because we're building assistive systems that, according to user feedback, can be very helpful for them in their daily lives," Tseng said.

Tseng also works on improving state-of-the-art algorithms for specific tasks that help people who are blind or with low vision.

One is a few-shot algorithm—an algorithm that only needs a few examples of a type of content—to identify and potentially obscure private parts of images, such as a prescription pill bottle or a pregnancy test.

This work will allow BLV people who show images to others to choose what private information others can see.

Tseng said that she appreciates working with Gurari. "She's great at guiding us to find what we're passionate about, what our research goals are, and how to make a good plan for the whole PhD," she said.

Jarek Reynolds, first-year PhD student

Jarek Reynolds' research involves developing datasets and algorithms that help BLV people better navigate their surroundings in their day-to-day lives, as well as protect their digital privacy.

Reynolds started by building a "salient object" dataset, which finds the most important objects in images taken by BLV photographers. He is now working on a privacy dataset, similar to Tseng's, but working on videos instead of images.

"Not only are you helping a demographic of people that are often marginalized, but the datasets and algorithms stem from authentic use cases, which makes the problems that much more interesting," he said.

Reynolds said he appreciates the brilliance and collaboration of the group.

"Dr. Gurari has a really good grasp of each of our strengths and weaknesses and how we can complement one another," he said.

Reynolds said that to be successful in a PhD, candidates should prepare to be humbled and take time to appreciate the journey.

"If there's something you would otherwise see as bad or a setback, that's not always the case. Every problem is an opportunity in disguise,'" he said.

Josh Myers-Dean, third-year PhD student

Josh Myers-Dean is making image editing tools easier to use by improving image segmentation, which entails breaking an image into discrete parts.

"I'm not working on generating fancy images, but, say, if you wanted to select a dog in a picture, I want to make that as easy as possible," he said.

Myers-Dean works specifically on interactive segmentation, which allows a model to determine if a user wants to segment a single object into its subparts, such as segmenting a chair into a back, a seat and legs, or finding the label part of a container of prescription pills. This could be useful for digital privacy, visual accessibility, and also creative applications such as image editing.

Myers-Dean received a National Science Foundation Graduate Research Fellowship, which he said has been an amazing advantage to his PhD journey by allowing him to focus on his research.

"Danna is great," Myers-Dean said. "She's super supportive. She helps keep me grounded and filter out the signal from the noise. She is also just a really kind person, and she's made me a better writer."

Myers-Dean recommended that anyone pursuing a PhD should try to focus on solving “problem A” before getting too excited about working on “problem B.”

"Try not to think too long on the horizon. Take the steps one at a time," he said.

Neelima Prasad, first-year PhD student

Neelima Prasad is researching how to track fast-moving objects in video footage. This involves self-supervised machine learning, which learns how to make sense of the data itself, rather than having already labeled data provided.

"It's actually a pretty hard problem," Prasad explained. "Let's say we have a soccer player. From one frame of a video to the next, they could be in a very different location, or they could go behind a goalie or a cameraman. How are we able to track them consistently over time?"

This problem has applications across many fields, including sports, autonomous driving and investigating natural phenomena, from the movements of bees to marine life.

Prasad said it's wonderful being a part of the group.

"Everyone has been so inclusive and supportive." she said. "We're all working on various aspects of computer vision. It's like we're all looking at different parts of the same elephant. I can talk to anybody about what I'm working on, and they are able to help me."

When asked what to consider when thinking about a PhD, Prasad said to have passion and some idea of where you want to end up.

"If you feel like there's something deeper you want to explore, a PhD is a great avenue to do that," she said.

Nick Cooper, first-year PhD student

Nick Cooper is working on untangling and demystifying deep learning classifiers.

"We send these neural networks into the field and when they come back, they can do amazing things, but what exactly have they learned? How are they doing it?" he said.

Deep neural networks, though able to perform very impressive feats, are often clunky, requiring vast amounts of electricity and time. This is because in order to "learn" something, they need to make many wrong choices.

As a metaphor, imagine taking a multiple choice test with no prior knowledge. You can take the test as many times as you wish and will be graded each time. Eventually, you can get an excellent score through trial and error, but this takes much more time and energy than if you were able to get the right answer the first time.

Cooper's research seeks to take these large, burdensome models that eventually learn how to provide correct answers, and streamline them into more compressed models that people can easily understand.

Traditional

White

AI Could Change How Blind People See the World

Anonymous — Mon, 10 Jul 2023 21:36:53 +0000

AI Could Change How Blind People See the World Anonymous (not verified) Mon, 07/10/2023 - 15:36

Danna Gurari, assistant professor of computer science at CU �鶹ӰԺ, shares her perspective on the promise and pitfalls of generative AI for blind users as the field explodes in popularity. External link to a WIRED magazine article.

Traditional

White

Keeping the unseen safe: Improving digital privacy for blind people

Anonymous — Fri, 19 Nov 2021 18:19:51 +0000

Keeping the unseen safe: Improving digital privacy for blind people Anonymous (not verified) Fri, 11/19/2021 - 11:19

Grace Wilson

Associate Professor Danna Gurari

Blind people, like sighted people, post on Instagram, swipe on Tinder, and text photos of their children to a group chat. They also use photos to learn about their visual surroundings.

Blind users often share images with identification software such as Microsoft's Seeing AI, Be My Eyes and TapTapSee to learn about their visual surroundings. There's a high demand too. Seeing AI, for instance, has been used over 20 million times.

When blind people share photos, however, there is an added risk that they could unknowingly capture information considered private, such as a pregnancy test or a return address.

To Assistant Professor Danna Gurari, this shouldn't have to be a concern.

Gurari, the founding director of the Image and Video Computing group in the Department of Computer Science, is part of a cross-institutional team that has been awarded over $1 million through a Safe and Trustworthy Cyberspace (SaTC) grant from the National Science Foundation to study the issue.

Currently, blind people must either trust friends or family members to vet their images for private information before sharing publicly, which can have social repercussions of its own. Or they can accept the risk to their privacy when they post.

The goal of the team's four-year interdisciplinary project is to create a novel system that can alert users when private information is present in an image and, if the blind person wants to, obscure it.

Working with human-centered computing expert Leah Findlater from the University of Washington and privacy expert Yang Wang from the University of Illinois at Urbana-Champaign, Gurari's group is leading the automatic analysis of images for the project. Their goal is to turn the desires of users and theories of private information into actionable knowledge.

This comes with a number of challenges, both technical and philosophical.

Because AI makes mistakes, you have to be careful how certain you make an analysis sound.

"We really want to endow the appropriate level of trust but also give decision-making power," Gurari said.

The Image and Video Computing group is creating ways to share what private information might be present in an image and let the user decide to use the image as-is, discard it, or obscure the private information and then share it.

The other problem to solve for Gurari's group is how to determine what the most prominent object in an image is and obscure everything else.

Because blind people often share photos for object identification, this feature could reduce the amount of private information introduced during this straight-forward task.

Illustration of envisioned user interaction pipeline for empowering users to safeguard private content in their pictures and videos. (a) For the general use case, the tool will notify the user about what private content is detected and then provide a choice to either discard the media, share it as-is, or share an edited version where private content (teel mask overlaid on image) is obfuscated. (b) For the scenario where a user wants assistance to learn about an object, the tool will share an edited version with all content outside of the foreground object (teel mask overlaid on image) obfuscated.

Gurari's team will be focusing on creating algorithms robust enough to counteract image blur and other properties common for images taken by blind photographers. The team must also craft algorithms that don’t need to be trained on specific objects to see them as important.

This object identification riddle is one that has haunted much of computer vision's history, termed the "long-tail problem" for the graph it produces.

Computers usually have very low levels of object detection accuracy until they have been trained on thousands of images, but here they must understand the significance of an object in only a few frames.

And, as with other assistive technologies, the benefits of these algorithms could go far beyond their original purpose. From product photography, which is built on isolating prominent objects, to alerting sighted users of private information they didn't notice, the project has great potential benefit for building a safe and trustworthy cyberspace for all.

Blind people, like sighted people, publicly share photos, but have limited options for checking if they've captured something they consider private. Assistant Professor Danna Gurari, as a co-lead for an over-$1 million NSF grant, is working to change this.

Traditional

White