The Center for Energy Smart Electronic Systems, established in 2011 with NSF funding as an Industry/University Cooperative Research Center, or I/UCRC, enters its third phase with sites at the University of Texas at Arlington as well as Villanova University in addition to Binghamton. The center’s 15 industry partners include Microsoft, Meta (Facebook), Bloomberg, Verizon, Corning, NVIDIA and Honeywell.

Bahgat Sammakia, vice president for research at Binghamton, serves as the director of the center, known as ES2. He notes that the three campuses and the companies they work with have made significant advancements in establishing methods that allow data centers to operate as dynamic, self-sensing and regulating systems that are predictable and verified in real time.

“We are doing this at every level of thermal management, from the chip to the data center, using AI and neural networks to optimize energy consumption at a multitude of scales,” Sammakia says.

There are just 83 I/UCRCs across the country. To date, NSF has provided the center with nearly $4 million in funding.

Data centers are among the most energy-intensive types of buildings and account for an estimated 2 percent of the country’s total energy use, according to the U.S. Department of Energy.

ES2 aims to help reduce that number — or to keep it in check — as data center usage continues to grow, in part by minimizing energy waste and recovering waste heat generated by data centers. In Phase III, ES2’s mission will expand to include edge computing, incorporation of renewable energy sources and cooling high heat-flux data centers as well as finding innovative ways to reduce the environmental impact of the data center ecosystem.

ES2’s research infrastructure across the three university sites includes:

• A one-of-a kind data center research laboratory and thermal lab at Binghamton.
• An electronics cooling lab, including a state-of-the-art immersion cooling facility, two data center lab facilities and an engineering research lab at UT-Arlington.
• An advanced liquid cooling laboratory for high-powered data center equipment and a high-performance computing laboratory at Villanova.

These facilities enable ES2 researchers to build, test and verify models and simulations at device, component and room scales.

NVIDIA, which joined ES2 in 2020, has collaborated with researchers at all three university sites. “ES2’s expansive facilities and expertise have enabled us to achieve many of our research objectives,” says Vladimir Troy, vice president of enterprise software at NVIDIA. “Several ES2 student interns proved to be such strong contributors that we hired them as employees upon completion of their Ph.D. studies.”

Dereje Agonafer, presidential distinguished professor and ES2 site director at UT-Arlington, says he’s especially pleased that founding member Meta will participate in Phase III. “Since the program started, we have been able to expand our lab facilities significantly, including new liquid and immersion cooling labs,” Agonafer says. “In addition to doctoral and master’s students, many undergraduates have taken advantage of the facilities as well in activities such as honors program and senior design projects.”

In the center’s first decade, 40 students earned doctorates while conducting research through ES2; another 65 students received master’s degrees. Most of them are now engaged in R&D activities at leading companies across the globe. With NSF support, ES2 has provided opportunities for undergraduates, teachers and other targeted groups to gain meaningful research experiences.

In addition, ES2-affiliated experts published more than 200 papers and filed 16 patents and invention disclosures during those 10 years.

That’s proof that ES2 has given faculty and student researchers opportunities to work on significant and timely research, says Al Ortega, James R. Birle Professor of Energy Technology and site director at Villanova.

“We are thrilled that the ES2 Center has helped our team become widely known experts in advanced data center cooling technologies and the fundamental science of convective cooling in single and two-phase flows,” Ortega says. “We are equally proud that the Villanova team has become the lead site in the area of sustainability by our research in energy and water usage efficiency and in waste energy recovery.”

ES2 researchers have new targets — items they see as the three major needs of modern data centers — as they look ahead to the next five years. They aim to:

• Automate operations that guarantee performance, reliability and availability with optimized energy consumption via energy-efficient improvements.
• Address the thermal challenges inherent in modern IT equipment that have increasingly large heat loads in ever smaller footprints, requiring liquid cooling in various forms, as well as data centers that rely entirely or partly on air cooling.
• Improve the sustainability of data centers through the use of renewable energy, stored energy, reduced water needs and harvesting waste heat.

Future Facilities has been a center member since 2012. Mark Seymour, chief technology officer, says ES2 research into the thermal performance of data centers continues to influence product development and the advice the company gives to customers.

“The ES2 data center has proven to be a highly valuable asset,” Seymour says. “We plan to take advantage of it on a regular basis to educate our clients and prospective clients by showcasing our products. In particular, we can highlight how the products help data center operators leverage the opportunities of energy-smart electronic systems.”

The National Science Foundation’s CyberCorps Scholarship for Service (SFS) program is designed to recruit and train the next generation of information technology experts and security managers to meet the needs of federal, state, local and tribal governments. In return for their scholarships, recipients agree to work after graduation in government cybersecurity positions for a period equal to the length of their scholarships.

At Binghamton, the SFS program will be overseen by faculty members from the Department of Computer Science and the Department of Electrical and Computer Engineering at the Thomas J. Watson College of Engineering and Applied Science. If it is deemed a success, the NSF could award further funding.

There were about 465,000 open positions in cybersecurity nationwide in 2021, according to the tech job-tracking database CyberSeek. The SFS program seeks to help fill the gap, with a special emphasis on attracting people of diverse backgrounds to the profession.

Binghamton University President Harvey Stenger sees the CyberCorps program as one that takes advantage of several of the campus’ core strengths.

“We know there’s a huge need in this field for highly trained experts,” he said. “Binghamton has a longstanding commitment to first-generation students and scholars from underrepresented minorities. We also have a robust set of course offerings and scholarship related to cybersecurity. It’s exciting to know that Binghamton will play a part in diversifying this essential workforce.”

Bahgat Sammakia, vice president for research, said Binghamton’s faculty members understand that research and teaching reinforce each other.

“This dynamic program will provide exceptional professional preparation for our students while also advancing Binghamton’s research related to information security,” he said. “I see in my own work how student contributions enable exciting discoveries and how strong mentorship can set up students to succeed in college and far beyond.”

In 2020, the National Security Agency and the Department of Homeland Security named Binghamton a National Center of Academic Excellence in Cyber Research (CAE-R) through 2025. The designation recognizes the work at the Center for Information Assurance and Cybersecurity (CIAC), led by computer science Associate Professor Ping Yang.

“Professor Yang’s vision, leadership and unwavering effort played an instrumental role in securing this grant,” said Professor Weiyi Meng, chair of the Computer Science Department. “It is probably the largest single grant in the history of the department, and it will have a big impact on the department, Watson College and Binghamton University for many years to come.”

Watson College Dean Krishnaswami “Hari” Srihari is proud of the faculty and staff who collaborated to seek the NSF funding, which is granted to fewer than 100 schools nationwide. He knows it will increase the visibility of Binghamton’s cybersecurity efforts in the academic community and the U.S. government.

“Our researchers and students are building the future by tackling our 21st-century problems head-on,” Srihari said. “Ensuring that our data remain safe is a key part of that future, and all of us will benefit from the knowledge shared here at Binghamton and Watson College.”

Serving as co-principal investigators on the NSF grant are Professor and Associate Chair Dmitry Ponomarev, Professor Kartik Gopalan and Associate Professor Aravind Prakash from the Computer Science Department, and Associate Professor Yu Chen from the ECE Department. Senior personnel include Distinguished Professor Jessica Fridrich (ECE), Professor Lijun Yin and Associate Professor Guanhua Yan (both CS).

Together, the faculty members cover a wide spectrum of research interests, from architectural support for security and software/systems security to steganography, artificial intelligence (AI)-based security and mobile security. The approach mirrors what Yang hopes will happen in government, corporate and nonprofit settings, especially when members of underserved communities earn their degrees and join the workforce.

“Building teams of cybersecurity professionals with a variety of skill sets brings different voices and perspectives to the table, which can help to improve our defense against a wider range of cyber threats,” Yang said.

The CyberCorps scholarship will focus at first on recruiting for Watson College graduate programs (including the 4+1 accelerated five-year program for earning both a bachelor’s and master’s degree), where a cybersecurity curriculum is already in place. The first students to receive the assistance could be enrolled in fall 2022.

“We are going to build up slowly, to start with maybe two students in the first year and then go to five or six new students in each of the subsequent years,” Ponomarev said. “We also will integrate this program with other research activities, so students can be involved in current research grants. There will be synergy, especially with two departments working together.”

The SFS program’s requirement of government service in exchange for funding students’ education is one way for the public sector to compete with corporations that often can offer higher salaries, especially for entry-level positions.

“This program addresses the needs in the government sector,” Gopalan said, “by having motivated students who apply, get training, get a feel for the environment where you work for the government and also get the satisfaction of protecting the nation’s infrastructure.”

The CIAC team knows that having capable cybersecurity professionals in government is no longer optional, not only because of the amount of personal data being stored but also because lawmakers need to understand and pass legislation that protects citizens.

“The government cannot fall behind, because the virtual world and the physical world are starting to merge tighter and tighter,” Chen said. “Facebook’s Metaverse, Apple and Google are building a new world — what some people consider ‘version three’ of the internet. Private companies in the high-tech sector are preparing for this, and the government needs to be aware of those innovations.”

Prakash agreed. “Government professionals may need to take an adversarial position in some cases, and those battles are difficult if there are no professionals in that area,” he said, “Unlike industry professionals, government professionals have a fiduciary duty to the people of the United States and not to the shareholders of a particular company. Without talented individuals in the government, cybersecurity-related lawmaking and enforcement will greatly suffer. This grant helps fill this critical need.”

Adiel Felsen, a junior dual majoring in computer science and mathematics, studies machine learning.

“Machines make mistakes but they make mistakes less often, especially if they’re trained properly,” Felsen says. “And there can be a lot of dangers in that if you start fully relying on only machines, but I really think machine learning is a great way forward for science in general.”

He has always had an interest in doing research, which played a part in his attraction to Binghamton and the Freshman Research Immersion program.

“I was interested before I went into the program, because the program is part of what attracted me to Binghamton,” Felsen says. “But it definitely allowed me to learn about topics that I wouldn’t otherwise learn until maybe senior year of school.”

FRI helped Felsen develop an understanding of data and machine learning. He categorizes data as anything from random images to letters and numbers. As a part of FRI, he worked on computer vision, a field of study where researchers teach computers how to complete simple tasks such as determining whether an image shows a cat or dog.

During his sophomore year, Felsen returned to FRI as a teacher assistant. In that role, Felsen not only assisted other students in understanding the basics of how to conduct research, he also solidified his own love for research.

Now working under Kenneth Chiu, associate professor of computer science; and Frank Lu, assistant professor of biomedical engineering, Felsen conducts research on biomedical imaging. He and his lab colleagues segment nuclei from cancer cells for SRS imaging.

SRS imaging, or stimulated Raman scattering microscopy, provides a detailed view of a cell. The method does not require the cell to be stained or dyed. Although it’s very fast, SRS imaging does not capture good contrast of DNA, and in biomedical imaging researchers need to be able to detect cell nuclei. Through machine learning algorithms, Felsen and his colleagues can improve the images’ contrast and see the nuclei.

One of Felsen’s mentors describes him as dynamic. “He has persistence and is always exploring new ideas,” Chiu says. “Those are the key things for research.”

Felsen will continue to develop his understanding of data and its many uses this summer as an intern at Mars Inc. He’ll work on projects dealing with data science for the chocolate company.

Felsen does not allow himself to be frustrated by minor setbacks because of what he sees as the impact of his work.

“Our research has the potential to help speed up the process of brain surgery, and it likely has a long way to go before it is actually implemented in the operating room,” Felsen says. “However, our research demonstrates that machine learning algorithms can improve the contrast of SRS images to look similar to stained cells. Hopefully, other researchers will see this promise and will be inspired to pursue similar work.”

Wenna Duan, a doctoral student in computer science, uses magnetization transfer rate (MTR) as a visual biomarker for brain tissue health.

MTR is a measurement most commonly used in magnetic resonance imaging (MRI) when looking at the brain. An MRI shoots energy into tissue cells, disorienting them, and then is turned off. MTR measures the amount of time the tissue cells take to dissipate the energy and reorient themselves.

Duan determined that brain tissue, specifically white matter, dissipates the energy more slowly in a brain that is more likely to suffer from Alzheimer’s. By comparing the MTR of brains suffering from different stages of Alzheimer’s with undiagnosed scans, doctors would be able to diagnose before the patient suffers from symptoms such as memory loss.

It’s a potentially vital development in the fight against Alzheimer’s disease, which the Centers for Disease Control and Prevention reported led to the deaths of more than 120,000 Americans in 2017.

The International Society for Magnetic Resonance in Medicine (ISMRM) accepted Duan’s research for its 2019 annual conference, where it received a magna cum laude award, which is given to the top 15% of thousands of projects.

The process of comparing scans was not a quick one. Duan had to create a template made up from the average of thousands of scans from a specific stage of Alzheimer’s. Then, because brain shape and size varies, she had to manipulate the undiagnosed scans, one by one, to fit the size of the template.

“The most challenging part of it all was in the preprocessing, especially because I had to teach myself the science behind the data to use it,” Duan says. “It took around two months to process all of the brain scans, and there were some very late nights.”

The brain data was collected as both longitudinal, where one subject was observed over several years, and cross-sectional, where one time-point was observed in different stages of Alzheimer’s, by the Cardiovascular Health Study at the University of Pittsburgh.

Weiying Dai, an assistant professor of computer science who previously taught at the University of Pittsburgh, supervised Duan’s research at Binghamton and provided the database. Dai says Duan’s research has the potential to make big changes in the clinical field.

“Her research can also help to identify potential treatment groups when a new drug comes to play as we do not currently have a cure for Alzeihmer’s,” Dai says. “Her work is clearly application oriented. If successful, she will make a huge contribution to both method development and clinical application.”

Duan is continuing her research with hopes of reconfirming her findings until this process is used regularly to help patients.

Her collaboration with Dai began when she took a machine learning course Dai taught.

“I did research with her and didn’t know it was all MR related, but when I turned in the MR project, she introduced how significant those findings are in the clinical area,” Duan says. “I thought, ‘You are really impacting somebody,’ so I got very excited and I decided to continue with that.”

Seden Akcinaroglu analyzes trends in domestic terrorist attacks to predict when they will occur in the future.

Akcinaroglu, an assistant professor of political science at Binghamton, specializes in international conflict and rivalries. In the past, she has focused on three general areas: civil wars, international rivalries and terrorism. Her interdisciplinary project with Binghamton University computer scientists homes in on the relationships between major events in a country and its terrorist attacks.

“We are trying to understand, in a very dynamic way, how we can actually predict some terrorist attacks,” Akcinaroglu says. “Right now, a lot of the studies are very static.”

Yu David Liu, an assistant professor of computer science, says Akcinaroglu’s dynamic research direction is “challenging but essential, considering the fast pace of global events.”

He says the project uses public sources such as online newspapers and databases. “It then performs data analysis to correlate individual factors and the historical attack data,” Liu says. “Prediction is made based on findings of these correlations from historical data.”

Liu says computers streamline the data collection and analysis so predictions can be made quickly.

Previously, Akcinaroglu worked to unravel the effect of natural disasters, economic status and militarized disputes on terrorist attacks. “When states are engaged in militarized disputes with other states, terrorist attacks increase,” she says. “The terrorists find it as an opportunity to attack.”

The same is true, Akcinaroglu says, for cases in which a country is weakened economically. When natural disasters hit, however, terrorist attacks tend to decrease.

“It seems that terrorists do care about reputation to a certain extent,” Akcinaroglu says, “so during times of natural disasters, they don’t want to alienate supporters.”

These findings challenge how terrorist organizations are normally perceived to operate. “For us, any atrocity can be attributed to a terrorist organization. The fact that they kill civilians is basically telling us that they don’t care,” Akcinaroglu says. “But they do care about reputation, and some of them care more than the others.”

Her current project uses a complex model to encompass as many factors as possible and look at how those factors affect terrorist organizations’ decisions to attack.

Akcinaroglu, who is from Turkey, says her interest in researching terrorist attacks began when the Kurdistan Workers’ Party (PKK), a rebel organization in Turkey, announced it would not attack the country following the devastating 1999 earthquake. Akcinaroglu says she saw the same thing occur in Indonesia later, and it made her curious about terrorist organizations’ hesitation to attack.

Vukasinovic, a Binghamton University junior studying computer science and neuroscience, recently took a semester off to address some health concerns. When he recovered more quickly than expected, he did what many young scientists and engineers dream of doing: He created his own company.

Now he’s CEO of a start-up called Emozia with 10 employees, many of them also Binghamton students. “Entrepreneurship is a quintessential aspect of our education,” Vukasinovic says. “I’m 21. I haven’t even had a real job. And suddenly you’re thrown into a whirlwind.”

Scott Hancock, director of IP management and licensing at Binghamton, has seen rising interest in entrepreneurship among students and recent graduates. Vukasinovic, he says, is among the most indefatigable of the bunch. “He’s relentlessly upbeat,” Hancock says. “He’s fearless and not afraid to take chances in pursuit of his passion.”

Vukasinovic smiles broadly as he describes building his network, beginning on campus and continuing with a New York City meeting of the nonprofit StartOut, which fosters LGBT entrepreneurs. From there, it was on to the Founder Institute, which bills itself as the world’s largest start-up accelerator.

Emozia, Vukasinovic explains, will harvest data via a user’s phone to “learn” whether he or she is feeling happy, sad, irritable, stressed or tired. Sensors in the phone will do some of the work, in concert with data from the user’s apps and calendar. Emozia will then share this information with service providers and friends selected by the user.

The result? Upbeat music starts playing when she gets in the car after a miserable day in the office. The lights dim automatically when she’s getting ready to go to sleep. Her best friend gets an alert when she’s feeling down.

“Obviously, we are pushing boundaries,” says Vukasinovic, who also sees applications in gaming. “Our company is based on trust. This doesn’t work unless people trust us with their data. And users will control how information is shared.”

Facebook, he notes, makes money by selling information about users to advertisers. Emozia is different, at least in his view, because third-party service providers will have to convince users that they offer enough value that they should have access to personal information.

Vukasinovic would love to see Emozia grow into the next dot-com sensation. But he doesn’t fear failure. In fact, he’s already talking about what things will be like when he starts another company.

“What I’ve already learned,” he says, “is so amazingly valuable.”

A Binghamton University researcher aims to slash the energy used by computing systems ranging from smart phones to data centers.

A new $450,000 grant from the National Science Foundation (NSF) will help launch Timothy Normand Miller’s ambitious new “introspective computing” project. The funding comes from the Faculty Early Career Development (CAREER) Program, which awards the NSF’s most prestigious grants in support of new researchers.

Miller said manufacturing variations present a severe — and worsening — challenge for computer systems. Semiconductor chips produced by the same factory do not perform identically, which means systems are designed to accommodate the worst chips.

He envisions using machine learning to predict a chip’s performance. “When you bring a chip online for the first time, it will monitor itself and dynamically adjust its own behavior,” Miller said. “The chip will make sure it does the right thing on its own. That would improve not only energy efficiency but also the lifespan of the chip.”

What makes that such a big deal? Energy efficiency in chips is critical to extending the operation time of battery-powered devices, and it’s a huge factor in the cost of running larger systems, including data centers.

Current power-management solutions focus on large systems, said Miller, an assistant professor of computer science. His plan is to work at the chip level, offering more sophisticated control and a means of scaling the solution up to address systems of a variety of sizes.

Engineers assume worst-case conditions in designing computer systems, Miller said. These assumptions are built into a “guard band” that ensures the system will work for 10 years even with the worst possible chip from the factory, even if the voltage is too low and even when the temperature is too high.

“That safety margin accounts for about 70 percent of the energy used by a modern microprocessor,” Miller said. “What we want is to have the safety margin be the minimum it has to be so that the circuit will operate reliably.”

With a narrower guard band, he said, a chip could be 2.5 times faster or deliver the same performance using only a third as much energy. And with machine learning, a chip can reach an ideal tradeoff between performance and reliability, adjusting continuously over its lifespan.

Miller, who joined Binghamton’s faculty in 2012 after earning a doctorate at the Ohio State University, worked in private industry for more than 15 years.

He said other disciplines — cognitive science, linguistics and experimental psychology, in particular — inspired this direction in his research. The patterns of thought and engineering habits he cultivated while working in the private sector continue to help him, too. “I learned a lot from industry about doing the engineering part of this job,” he said. Miller said he often writes code to put ideas to the test quickly and enjoys helping students debug programs.

He is developing a short course for high school students that offers an analogy to introspective computing. Students will be challenged to come up with the careful scheduling required to operate a 22nd-century smart home with a restrictive amount of solar power and a set of activities that must be accomplished each day.

The environmentally friendly thing to do is to have each sensor powered independently, perhaps by a small solar panel. But it gets expensive to attach a generator of any kind to each sensor that’s large enough to handle peak energy needs, when it won’t need that energy much of the time.

The network could use smaller, and cheaper, solar panels or other generators if the sensors could share their excess energy with their neighbors and ask for power when they need it, but all those wires are expensive and inefficient.

The network could use a central power source, but that’s vulnerable to failure, sabotage or even attack, in the case of military installations. A distributed network provides more reliability but is more difficult to coordinate.

Ting Zhu, an assistant professor of computer science at Binghamton University, plans to tackle this challenge. He and his graduate students will develop the hardware and software to allow sensors in a wireless network to share energy.

“We built a prototype that can distribute energy over wires,” Zhu said. Now he hopes to take the next step. The team recently received a three-year, $450,000 grant from the National Science Foundation. Zhu provided the proof of concept last year with a wire-based network.

Here’s what the Binghamton team aims to do:

• Create a network that monitors energy needs and available energy for each sensor.
• Establish a protocol so a sensor can ask its neighbors for spare power.
• Develop a way for each sensor to send and receive energy at a range of up to 10 meters with minimal loss of energy.

“The major problem of wireless energy transfer is efficiency,” Zhu said. The initial work in the field was done by Marin Solijacic and a Massachusetts Institute of Technology team, which wirelessly powered a 60-watt bulb at a distance of 2 meters with 40 percent efficiency in 2007.

In wireless power transmission, a magnetic field oscillates. That, in turn, causes other nearby magnetic fields to oscillate, generating electricity.

An Intel-sponsored team achieved 80 percent efficiency over two meters in 2009, with devices the size of basketballs. And a Stanford University team announced a project last year to develop high-power transfers — enough to recharge an electronic vehicle while in motion. But those projects are limited to 2-meter transfers. What about 10 meters?

“It’s a complicated hardware design,” Zhu admitted. “This requires deep understanding of wireless energy transfer principles.”

It is work that Solijacic sees as a logical application of the pure research he started six years ago — after being awoken by the beep of a cell phone’s low-battery warning. The MIT-affiliated spinoff company he founded, WiTricity Corp., envisions using magnetic resonance to power robots, electric vehicles, household electronics — and sensor networks

Each point in the network Zhu is devising would include several elements: an energy generator, most likely a solar panel; a capacitor to store and discharge energy efficiently; a magnetic resonance device to send and receive energy; a sensor; and a means to broadcast its information to a central node.

The goal is to do that for $1 a device. At that price, the hardware to monitor a typical grocery store-sized space — 100,000 square feet — would cost about $100. At the moment, Zhu said, the cost is more like $300 a device.

At $1 a device, you could find someday that sensor networks initially designed for industrial or military use could be protecting your back yard, and maybe even watering the lawn.

“Real-time data is very dynamic and unpredictable,” says Kyoung-Don Kang, associate professor of computer science at Binghamton University. For example, Kang explains, a traffic-monitoring system might not see much activity at midnight Sunday, but it will generate tremendous amounts of data during Monday’s morning rush hour. That data could slow down the processing system, right when it’s needed most. “It is very challenging to process this data in a timely manner,” he says.

When real-time computing fails, it can compromise safety or lead to financial loss.  That’s why Kang is working to make this fast-paced data processing more efficient, with help from a National Science Foundation grant of nearly $250,000.

“It is an important research area, especially at this point when we have lots of critical systems depending on continuous streams of real-time data from zillions of sensors deployed in the environment,” says Sang H. Son, a computer scientist at the University of Virginia.

Why not just design systems that are capable of processing massive amounts of data all the time? It’s not practical, Kang says, because most of the time a system will need to process only sparse amounts of data — and when it sits idle, that’s a waste of resources. And data is always increasing in volume, so even today’s top-notch system will be outpaced eventually.

Kang says the key to using real-time data applications is to cut your losses. If the amount of data is more than the system can handle, then some of it must be dropped, he says: “Some data is more important than others.”

That’s why he’ll be developing algorithms and software solutions that process the most vital data stream first. Kang will use simple yet powerful rules to prioritize some operations over others — for instance, if an input data stream is important, then the query processing output from that data stream is likely to be important as well — to build more efficient load-shedding and continuous query processing techniques. This approach can be applied to detect important events, such as unusual traffic patterns or homeland security issues, in real time.

More efficient processing of real-time data could one day enable other technological advances, including directing intelligent transportation or managing green buildings and smart grids. “There are many potential applications,” Kang says. “The challenge is being ready for anything.”

Last summer, senior Mario Alexandre, a computer engineering major, was paired with a biology major through a $1.4 million, four-year grant from the Howard Hughes Medical Institute, or HHMI. The grant supports projects that solve problems in the life sciences; undergraduates work with faculty mentors on original research.

The biologists in Alexandre’s group worked with flow cell systems for multiple bacteria species to model their behavior. A few hundred images were taken in a 3D space, and were then sliced into 2D images. Alexandre developed software to analyze these images for different properties and determine specific characteristics, such as density, of the bacteria in question.

The research, which could translate into increased effectiveness for medications and vaccinations, revealed that more bacteria were on the outside than inside of the biofilms. “This means medicine can only penetrate halfway through and can only kill the first layer of cells,” Alexandre said. “You’ll only weaken the actual infection; you won’t beat it.”

He continues to work with the analysis generated during the summer with plans to pass the research back to the biologists. Alexandre is developing a user interface that will allow biologists to run tests with speedy results and view directories of all the images.

“The experience introduces Mario to an area of study that is not usually in the engineering curriculum,” said Anna Tan-Wilson, a distinguished teaching professor and the HHMI program director. “He is doing research in the life sciences. This kind of interdisciplinary work is seeing a lot of growth within engineering research, and it expands Mario’s career opportunities.”

Alexandre said the project helped him to think differently about his own field. “Knowing that image processing, a component of both computer science and electrical engineering, can greatly aid the development of biology was amazing,” he said. “I didn’t know you could do that.”

One day, Yin describes how facial-recognition software could be used to treat children with autism. Another time, he builds a digital 3D likeness of himself from just two photographs. During a third visit, a graduate student advances the slides in a PowerPoint presentation using only his eyes. When children come by, Yin shows them how to create a brief animated clip using his software just by speaking into a microphone.

Some researchers focus on one topic, probing deeper and deeper over a period of decades until they have an exhaustive knowledge of the challenges and solutions related to it. Yin, on the other hand, constantly finds new applications for what he knows. His ideas may one day advance fields as diverse as education, healthcare, entertainment and homeland security.

“Our research is motivated,” he says, “by a desire to improve computers to provide something good for our society. I try to use my sophisticated technology to make computers easier to use by the nontechnical person.”

Yin, a computer scientist who also studied electrical engineering, says his fundamentally interdisciplinary work relies on psychology and mathematics as well. He speaks about the possibilities for technology to improve robotics and plastic surgery as though they’re intrinsically related. That’s because, to his mind at least, they are.

Human computing

Yin wants to enable computers to understand inputs from humans that go beyond the traditional keyboard and mouse.

“Our research in computer graphics and computer vision tries to make using computers easier,”  he says.  “Can we find a more comfortable, intuitive and intelligent way to use the computer? It should feel like you’re talking to a friend. This could also help disabled people use computers the way everyone else does.”

Yin’s team has developed ways to provide information to the computer based on where a user is looking as well as through gestures or speech. One of the basic challenges in this area is “computer vision.”  That is, how can a simple webcam work more like the human eye? Can camera-captured data understand a real-world object? Can this data be used to “see” the user and “understand” what the user wants to do?

To some extent, that’s already possible. Witness one of Yin’s graduate students giving a PowerPoint presentation and using only his eyes to highlight content on various slides. When Yin demonstrated this technology for Air Force experts last year, the only hardware he brought was a webcam attached to a laptop computer.

Yin says the next step would be enabling the computer to recognize a user’s emotional state. He works with a well-established set of six basic emotions — anger, disgust, fear, joy, sadness and surprise — and is experimenting with different ways the computer can distinguish among them. Is there enough data in the way the lines around the eyes change? Could focusing on the user’s mouth provide sufficient clues? What happens if the user’s face is only partially visible, perhaps turned to one side?

“Computers only understand zeroes and ones,” Yin says. “Everything is about patterns. We want to find out how to recognize each emotion using only the most important features.”

He’s partnering with Binghamton University psychologist Peter Gerhardstein to explore ways this work could benefit children with autism. Many people with autism have difficulty interpreting others’ emotions; therapists sometimes use photographs of people to teach children how to understand when someone is happy or sad and so forth. Yin could produce not just photographs, but three-dimensional avatars that are able to display a range of emotions. Given the right pictures, Yin could even produce avatars of a child’s family members for use in this type of therapy.

Yin and Gerhardstein’s previous collaboration led to the creation of a 3D facial expression database, which includes 100 subjects with 2,500 facial expression models. The database is available at no cost to the nonprofit research community and has become a worldwide testbed for those working on related projects in fields such as biomedicine, law enforcement and computer science.

Artificial intelligence

Once Yin became more interested in human-computer interaction, he naturally grew more excited about the possibilities for artificial intelligence.

“We want not only to create a virtual-person model, we want to understand a real person’s emotions and feelings,” Yin says. “We want the computer to be able to understand how you feel, too. That’s hard, even harder than my other work.”

Imagine if a computer could understand when people are in pain. Some people’s gestures and facial expressions may change. Some may ask a doctor for help. But others — young children, for instance — cannot express themselves or are unable to speak for some reason. Yin wants to develop an algorithm that would enable a computer to determine when someone is in pain based only on a photograph.

Yin describes that healthcare application and, almost in the next breath, points out that the same system that could identify pain might also be used to figure out when someone is lying. Perhaps a computer could offer insights like the ones provided by Tim Roth’s character, Dr. Cal Lightman, on the television show Lie to Me. The fictional character is a psychologist with an expertise in tracking deception who often partners with law-enforcement agencies.

“This technology,” Yin says, “could help us to train the computer to do facial-recognition analysis in place of experts.”

A pragmatic approach

Yin may dream big, but he’s also mindful of the limitations imposed by the real world when it comes to his ideas.

His pragmatic approach to the challenges of computing in everyday life was critical to his role in developing the MPEG-4 standards now used in digital video when he was still in graduate school. The goal there was to save bandwidth by compressing raw data as much as possible without losing information or quality.

These days, he hopes to make it easier to identify suspects passing through security checkpoints at airports. But he knows that for such a security algorithm to be useful, a low-resolution camera must be able to do advanced detection work. There’s no way to bring his laboratory’s elaborate six-camera setup into every airport. And there’s no way to have each passenger pose at exactly the right distance from the camera to be identified.

Yin’s goal is to create a facial-recognition algorithm that would be able to pick a person out of a crowd, given only front and side photographs of the individual. And it has to work even if he or she passes a camera at another angle.

“Our current work,” he says, “uses a regular camera system to do this challenging job.”

Yu David Liu received a five-year, $448,641 grant from the NSF’s Faculty Early Career Development (CAREER) Program. The highly competitive grants support junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education and research.

Liu joined Binghamton’s faculty in 2008, after earning master’s and doctoral degrees in computer science from Johns Hopkins University. He also recently received a $50,000 grant from Google for a related research project.

Computers and electronic devices, ranging from smartphones to servers, consume a steadily growing amount of energy. In recent years, computer scientists have developed an interest in paring back this consumption, though generally they’ve approached the challenge through modifying hardware or perhaps operating systems. Liu plans to tackle the problem by considering how programmers can create more energy-efficient software.

“Saving energy is an activity that should come from many layers,” said Liu, who plans to build energy-related parameters into a programming language.

A change at that level would permit and encourage programmers to express their energy-saving intentions directly when software is developed. “Saving energy is often a trade-off,” Liu said. “Sometimes you’re willing to run your program slower so your cell phone battery can last longer.” For such settings — often specific to the nature of the applications — no automated algorithms know as much as programmers.

“Programs today are not just 50 lines of code,” Liu said. They have often grown to be thousands or even millions of lines long. He hopes to employ advanced programming language technologies known as “type systems” to answer questions such as “What is the energy-consumption pattern of a large program, given the consumption patterns of its fragments?” and “Do programmers have conflicted views of the energy-consumption patterns of their software?”

Energy-efficient solutions at the level of programming languages also enjoy a high degree of platform independence, meaning they can have an impact all along the spectrum from phones to servers. “In an era when new platforms are introduced every year,” Liu explained, “an approach that’s platform-independent would be beneficial because it can be applied more broadly.”

None of the mainstream computer languages supports energy-aware programming, he said. However, language designers often create a blueprint that can be extended. Java, for instance, could be extended as EnergyJava and remain 90 percent the same. Such moderate changes would make it possible for programmers to adopt it relatively easily.

There isn’t much history in this area, Liu said, so it’s hard to say how quickly industry will react to the development of an energy-efficient language. However, new language designs have the potential to influence how millions of programmers think.

“I think every researcher wants to make the world better, and we just put it into our own perspective,” he said. “Sometime in the future, every Computer Science 101 class may include a lecture or two on energy-aware programming. As an educator, I’m excited about helping to ensure that next-generation programmers are green-conscious from the beginning of their careers.”

Everything about the New Hampshire lake, from its blue-green algae to its water quality, is fodder for researchers. But what was learned about the lake wasn’t always filtering through to the people who live and play nearby.

Binghamton University’s Kenneth Chiu and his students set out to change that, putting their expertise in computer science to work for the good of the community. Chiu, his students and other collaborators created a website that allows people to harness tremendous amounts of information and learn what’s going on in those waters. In the process, both the computer scientists and the citizen scientists learned about designing technology for users in the real world.

When the Lake Sunapee Protective Association (LSPA) was founded in 1898, water-quality issues were tied to sawdust and trash in the lake and the level of the lake water. Today, the organization uses science to better advocate for the environmental vitality of the lake and its watershed. To that end, it built a water-quality buoy, which collects and transmits data every 10 minutes on weather and lake conditions.

“It quickly became clear that a buoy can generate an awful lot of data in a short period of time,” said June Fichter, executive director of the LSPA.

As part of its educational mission, the LSPA looked for a way to present that data to the public.

“Part of our mission is to explain it to kids in preschool, adults and everybody in between,” Fichter said. “How to appeal to all those people was tricky.”

The project got its start when Chiu and some colleagues proposed a project to engage nonprofit organizations in the design, development and deployment of advanced computer technology. It won support from the National Science Foundation, which wants nonprofits to learn how to use cutting-edge computer technology.

The website they created displays indicators such as water temperature, wind speed and dissolved oxygen, which can indicate the presence of organic matter and pollution.

Fichter said the project energized people in the Lake Sunapee group. “What’s exciting is that this will be easy to use, and it will draw people in to the science,” she said. “Of course, our ulterior motive is that it will be a new tool for them to learn about lakes and watersheds and how it all works. And, in the end, they as voters can influence public policy toward preserving watersheds.”

Chiu said the project provided valuable opportunities for instruction. “I think this was a good introduction for the students in how to interact with real users,” he added. “It gives them a head start.”

Chiu, whose research area is distributed systems, acknowledges the project didn’t push the envelope in terms of what’s possible in computer science. But when it comes to finding ways that his field can benefit society, the Lake Sunapee work was rewarding.

“This project provides a broad context for the research that I do,” he said. “It also provides a degree of reality check for computer science, in that it helps us understand better what people really want out of technology.”

In fact, the Lake Sunapee project deepened an interest in ecosystem modeling that Chiu will pursue with support from two grants.

Ecosystem modeling involves coming up with a set of equations and rules for how an ecosystem behaves. These models are often developed independently by different scientific communities. To answer questions relevant to society, these models need run together. That’s challenging from a computer science perspective because of differences among the models.

The National Science Foundation gave Chiu and his collaborators a grant of more than $1 million, with $238,000 going to Chiu. He and several colleagues have also received a grant from Amazon Web Services in Education to use its system to do ecosystem-modeling research in cloud computing.

Chiu used the example of a proposed golf course next to a lake to explain the promise of this work.

“Scientists know rain runoff from the golf course will affect water quality, but how much, and in exactly what way? Will it change the water color year-round, or just in late summer? Will it change the kind of fish in the lake? Well, if we can build a model of the lake in the computer, then we can simulate the lake,” he said. “We can build a simulated golf course, simulated fertilizer, have some simulated rain and so on.”