Future Video Synthesis with Object Motion Prediction Yue Wu, Rongrong Gao, Jaesik Park, Qifeng Chen CVPR, 2020 Paper / arXiv Code. The most imposing limitation was that the end system cannot require users to download a separate application, and instead ONLY use what would be available in modern web browsers. This gives you an idea of what an end-to-end system could look like, but without the rtc-shapeshifter- or rtc-deflectometry-specific details. CampusWire will be staffed at specific times, when a member of the team will be answering questions (existing and new). Each homework consists of a coding and a technical writeup. Applicants should hold a 4-year bachelor's degree (or equivalent). Email / GitHub / LinkedIn. When paired with some JavaScript I wrote for generating sinusoidal patterns on the K1’s display, he can generate any number of periodic image patterns on the display, use WebRTC to record image captures of the morphed pattern, transmit them to the processing server, and see the phase mapping results in real-time. rtc-deflectometry was demonstrated on the Kokomo sample glass tiles, on decorative pieces we acquired for measurement purposes, and on various other objects (even those not strictly made of glass) that exhibit specular reflection. Tao Yue. Office hours are replaced with increased Campuswire activity on myside. PMD, for the unfamiliar, can be described as projecting light in varying structured patterns and using a camera element to perceive how a surface affects the reflection of the pattern. I obtained my Ph.D. in computer science from Northwestern University, where I worked on computational photography and computer vision with Oliver Cossairt in Computational Photography Lab . Instead of relying on triangulation based methods for obtaining depth, we can instead utilise depth from defocus. The device used for data capture was again an NVIDIA SHIELD K1 tablet. Homework is due and assigned on the dates below. You can resubmit up to three homework assignments that you received a failing grade for. Seam Carving and Lightfield Camera. This course will first cover the fundamentals of image sensing and modern cameras. I'm an assistant professor at Graduate School and Faculty of Information Science and Electrical Engineering, Kyushu University. My research interest stems from my deep fascination with upcyling. Your code must be pushed to your individual GitHub Classroom code repository, also at 11:59pm on the due date. Personal portfolio powered by Jekyll and GitHub Pages . rtc-shapeshifter is a WebRTC-based tool that expands upon a concept originally presented by Chia-Kai Yeh called Shape by Shifting. You can resubmit up to three homework assignments that you received a failing grade for. I am a Master student studying Computer Science at Northwestern University, IL, advised by Prof. Oliver Cossairt.I received my B.Eng. Many of the course materials are modified from the excellent class notes of similar courses offered in other schools by Shree Nayar, Marc Levoy, Jinwei Gu, Fredo Durand, and others. Welcome to Winston’s homepage! Computational Social Science research luncheons, Northwestern University. Computational illumination is used within the movie industry to render the performances of live actors into digital environments. During my time spent in Northwestern University’s Computational Photography Lab, I divided my attention between the mothballed handheld 3D scanner project and another project oriented around WebRTC. Web Application Waldo. they're used to gather information about the pages you visit and how many clicks you need to accomplish a task. My personal website. While I will not go into deep technical detail on his work, I included some slides from a presentation we held for one of the university’s scientific interest groups on October 19th, 2018: In short, Kai has been using the webrtc-perception framework to make it easier for him to recover surface normal maps with an off-the-shelf NVIDIA SHIELD K1 tablet though the use of photometric stereo measurement. I am currently taking the course CS101c: Computational Cameras with Prof. Katie Bouman. Before joining Northwestern, I spent one year (Oct. 2011 – Aug. 2012) as a Postdoctoral Researcher at Columbia University, under the … flash units) and area sources (e.g. Academic dishonesty will be dealt with as laid out in the student handbook. CampusWire—your first stop for questions and clarifications. In particular, Dr. Florian Willomitzer, the leading CPL post-doctoral researcher, was eager to measure some special glass tiles that we had in the lab. The instructors are extremely thankful to the researchers for making their notes available online. My aim was to develop an image capture framework that could be immediately usable for multiple ongoing research projects. Aug/2020: I successfully defended my M.S. Computational photography is the convergence of computer graphics, computer vision, optics, and imaging. Email / GitHub / Google Scholar / LinkedIn. This did threaten to constrain the potential capabilities somewhat, but also ensured a broader potential audience and subsequent use. Research. Grading: Homeworks 1 through 7 are each graded Pass/Fail. Project coming soon. GitHub is where people build software. In addition, the photography may involve (d) external illumination from point sources (e.g. Computational Photography . The next sections outline the goals of rtc-shapeshifter and rtc-deflectometry and how my colleagues are using webrtc-perception to achieve those goals. Analytics cookies. Unconference Breakout Sessions (4 consecutive slots of 30-min each). This includes free response answers and code. I gave a guest lecture to the CP Seminar course. For coding questions that involve your own code, please make a private thread that is only visibile to TA/Instructor. Students with a bachelor’s degree in a field other than CS are encouraged to apply, but to succeed in graduate-level CS courses, they must have prerequisite coursework or commensurate experience in object-oriented programming, data structures, algorithms, linear algebra, and statistics/probability. Our lab also looked at this project as a chance to create a system that could eventually be used by individuals outside of our laboratory, namely art curators and conservators, for historical or scientific documentation purposes. I received ME and PhD from Nara Institute of Science and Technology (NAIST) in 2016 and 2019, respectively. Enrollment is limited to 30 students. Here are three projects I implemented for the Computer Vision and Computational Photography course I took Fall 2015 at the University of Pennsylvania. Lunch Break. 28, Issue 7. Artificial Intelligence & Computational Photography - Haoban. I'm broadly interested in 3D-related computer vision research, including reconstruction, depth sensing, novel view synthesis, inverse graphics, computational photography, etc. You can sign up for the page at that link using the sign-up code 6624. Cheating & Academic Dishonesty: Do your own work. When and Where to Submit Assignments: A latex writeup report for each assignment must be submitted on Canvas For each assignment that you fail, your grade gets lowered by one letter. Tuesdays and Thursdays 1:00pm-2:20pm CT I am broadly interested in the interdisciplinary research of Computer Vision and Computer Graphics. "3D DiffuserCam: Single-Shot Compressive Lensless Imaging." We will then use this as a basis to explore recent topics in computational photography such as motion/defocus deblurring cameras, light field cameras, and computational illumination. Email / Google Scholar / Github We plan to stick closely to these grading guidelines, but some exceptions may be made for partial credit (e.g. I am now a computer vision engineer at Apple. Students should have experience with Python programming. Pieces commissioned by Tiffany usually bear artistic and historical relevance, but traditional surface measurement systems can be difficult to situate and leverage if the glass work is installed and immobile. We will be checking for code duplication. His work originally used DSLR cameras to get preliminary results and he switched to using an iPhone (with some special hardware) in its final form, which made it an interesting candidate for extension through webrtc-perception. Computational imaging stands in the crossroad of computer graphics, computer vision, and optics and sensors. Again, for a monocular method, depth from defocus (DfD) requires a comparison image. Nick Antipa*, Grace Kuo*, Ren Ng, and Laura Waller. We will then continue to explore more advanced topics in computer vision. I even got to do a bit of hand modeling for the feature’s preview image! The project “metapackage” is named webrtc-perception and is hosted over on GitHub. Computational Photography SIGGRAPH Course (Raskar & Tumblin), Computational Camera and Photography (Raskar, MIT), Digital and Computational Photography (Durand & Freeman, MIT), Computational Photography (Levoy & Wilburn, Stanford), Computational Photography (Belhumeur, Columbia), Computational Photography (Essa, Georgia Tech), Introduction to Visual Computing (Kutulakos, U of Toronto). Several applications and products already leverage WebRTC for video conferencing, gaming, media sharing, and other social applications, so it has benefited from steady growth and support since its introduction at the 2013 Google I/O developers conference. The Lytro Camera captures a 4D light field of a scene, enabling photographs to be digitally refocused after images are captured. 1. 1.3 Elements of Computational Photography Traditional film-like photography involves (a) a lens, (b) a 2D planar sensor and (c) a processor that converts sensed values into an image. \The Role of Niche Signals in Self-Organization in Society" Teaching Instructor of Record Computing Essentials for Social Scientists - Northwestern University - Summer 2018 Social Dynamics - University of Michigan - Fall 2013 Graduate Student Instructor WebRTC (RTC stands for Real-Time Communications) is a suite of APIs that enables the capture and transfer of video and audio content entirely through a web browser. Sep/2020: I started working at Dr. Vladlen Koltun's Intelligent Systems Lab at Intel. Participants are encouraged to … Computational Photography and Image Manipulation. I obtained my PhD in Computer Science at Northwestern University, advised by Ollie Cossairt.My research interests include computer vision and machine learning. # Computational Photography (ICCP), 2014 IEEE International Conference on # # hL and hH are the one dimenionsal filters designed by our optimization # bL and bH are the corresponding Chebysheve polynomials # t is the 3x3 McClellan transform matrix # directL and directH are the direct forms of the 2d filters The Python code converts the results of the computation into a format which can be transmitted to another, separate website designed to display (and make available, if necessary) the results. Its role is to overcome the limitations of traditional cameras, by combining imaging and computation to enable new and enhanced ways of capturing, representing, and … 28, Issue 7 in March 2020, and there is even a patent pending on this particular combined integration of PMD and mobile devices. Research. getUserMedia() and other MediaStream components simplify connecting to a client device. The client device, thanks to other MediaStream features, also permits the server to detect and choose which photography settings are important for that particular camera track (such as exposure time, ISO, white balance, focus distance, rear torch status, etc). Some developers and researchers have also used WebRTC to facilitate IoT applications, serve as the framework for hobbyist projects, and have integrated it into cutting-edge computer science and robotics research. Optical Society of America, 2017. rtc-deflectometry is a WebRTC-based tool that implements Phase Measuring Deflectometry (PMD) in order to optically measure surfaces that exhibit specular reflection. A good approach is to continually check in and push to GitHub as you work. Computational Optical Sensing and Imaging. Your coding must be correct, and your writeup must be clearly written (see latex template here: ) in order to receive a passing grade. Penalties include failing the class and can be more severe than that. The changing of light patterns requires some JavaScript and trigonometric acumen on the developers’ part, but the client merely needs to reload the webrtc-perception interface to get updated JavaScript code, and tweaks to server processing code are invisible to the client device. The design of webrtc-perception includes a capture website, a dedicated server for processing image data, and a results display website. Our work was presented at 2019’s AAAS conference and highlighted by AAAS on Science magazine’s website, as well as featured on Northwestern University’s Engineering News reel. Recent Projects. This system has made it far easier to perform surface measurements of painted works of art for the purposes of preservation and restoration. Each application is connected to specific active research projects in the Computational Photography Lab. To teach the fundamentals of modern camera architectures and give students hand-on experience acquiring, characterizing, and manipulating data captured using a modern camera platform. A barebones illustration of the webrtc-perception framework is shown in the following figure. Examples of application-specific code is contained within the “content” folder, while the metapackage itself serves as the issue tracker and documentation holder for all contained content. Who may apply? The featured implementations attempt to do this as close to real-time as possible, so that the user in control of the measurement client can evaluate the measurement process in a sort of feedback loop. (4 consecutive slots of … to ensure we cover many different topics. About Projects Resume Contact. This also confers some advantages, as operators can improve the processing code on the fly, change camera controls and presentation details on the respective websites, and fix issues without users needing to download or install any new files or update applications. The goal of this homework is to explore the focus properties of images captured by your Tegra device. since these projects are still used as homework assignments for the course. Special thanks to the NU Computational Photography Lab for the screenshot of Kai’s work currently serving as the project thumbnail. He can control various photography settings remotely, trigger image capture from the rear-facing camera (with the LED light enabled), clip on his polarizer, and automate processing and results generation…and see his results while capturing data. (5) Northwestern Neuroimaging and Applied Computational Anatomy (Lei Wang) 20 min (6) Michigan Institute for Data Science (Ivo Dinov), 20 min 12:00-1:00. degree in Software Engineering at Sichuan University in 2019, supervised by Prof. Jiancheng Lv.I’ve also attended the Summer workshop at National University of Singapore in Big Data & Cloud Computing with full scholarship. Lectures will also be recorded for those who cannot attend during scheduled class times. Computer Vision . I was a research intern at MSRA, supervised by Dr. Xun Guo. This project also leans on another library named aiortc to implement Python-based interaction with connecting clients via WebRTC and perform useful computation on images and other data gathered through use. The most recent code on github at 11:59pm on the due date is the code we will grade. Filters and Frequencies. I am actively working with deep neural networks for videos and image sequences. My research interests lie in Computer Vision, Deep Learning and Computational Photography. Computational photography combines plentiful low-cost computing, digital sensors, actuators, and lights to escape the limitations of traditional film-like methods. Mail: florian.schiffers (a) northwestern.edu It is a fairly tight schedule It offers a powerful tool to combine algorithms and sensing systems to outperform traditional sensors. EECS 395/495: Introduction to Computational Photography . My … Since joining the lab, under the guidance of Dr. Oliver Cossairt and Dr. Florian Willomitzer, he has been focusing on two practical applications of computer vision for scientific data collection. Specifically, I am interested in Vision and Language, 3D vision, Neural Rendering, Computational photography, Image & Video Understanding, AR/VR & Embodied AI. My research interests lie at the intersection of optics, computer vision, and computer graphics. The most recent submission in Canvas at that point, is the one we grade. Conferences: ICCP 2011, ICCP 2010, ICCP 2009, SIGGRAPH, SIGGRAPH Asia, CVPR, ICCV, ECCV, .. However, a requirement for a zoom session is to have an active Campuswire thread. For example, students will learn how to estimate scene depth from a sequence of captured images. studio lights). At present, two applications are featured in the metapackage: rtc-shapeshifter and rtc-deflectometry. If serious problem regarding an assignment arise, I am available for zoom session on an individual basis. The server handles gathering data from the client and performs application-specific computation on all the gathered data. webrtc-perception uses the WebRTC framework to establish a connection between a server and a client device in a seamless manner. If you are interested, please contact the instructor to discuss! Associate Professor Nanjing University School of Electronic Science and Technology Computational Sensing and Imaging Lab E-Mail: yuetao@nju.edu.cn Tao Yue received his B.S. The Nvidia Tegra Shield is an Android-based tablet that features a 5-megapixel camera with an easy to use camera API. These sample tiles have a particular surface shape that, if accurately captured, can be attributed to Kokomo’s specific roller table process. Save your images that you’ll use for the results and your report in png format. Programming Assignment 1 This assignment is intended to familiarize you with image filtering and frequency representations. 1:00-3:00. by 11:59pm on the due date. Computational Photography and Image Manipulation as a class is tought in many institutions with varying flavors. William Spies is an aspiring Roboticist and Research Scientist currently serving in the Computational Photography Lab at Northwestern University. During my time spent in Northwestern University’s Computational Photography Lab, I divided my attention between the mothballed handheld 3D scanner project and another project oriented around WebRTC.WebRTC (RTC stands for Real-Time Communications) is a suite of APIs that enables the capture and transfer of video and audio content entirely through a web browser. This course is the first in a two-part series that explores the emerging new field of Computational Photography. A-/B+, etc.). Florian Schiffers I started looking at WebRTC APIs in mid-2018 to determine if our lab could use such a technology as the basis for a new scientific data collection system. This iteration of the class makes use of material from the classes by James Tompkin, Ioannis Gkioulekas, Marc Pollefeys, and Alyosha Efros. I'm interested in computational photography, computer vision and machine learning. Optical Society of America, 2017. Florian Willomitzer The Computational Photography Lab is led by Prof. Oliver Cossairt, Associate Professor in the Department of Electrical Engineering and Computer Science at Northwestern University. The camera parameter could be aperture, exposure, focus, film speed or viewpoint. We use analytics cookies to understand how you use our websites so we can make them better, e.g. The server does all this through the use of Python and aiortc to connect with a client via WebRTC without needing to use a web browser itself. Computational Photography. Computational Optical Sensing and Imaging. Our results and a description of the work was featured in Optics Express Vol. Disclaimer. At other times, please pull together as a class and help each other, and we'll help soon. Hi there, My name is Wang, Zi-Hao (王子豪) and I go by Winston. So if you pass all seven assignments you get an A, if you fail one assignment you get a B, if you fail two you get a C, and so on. More than 50 million people use GitHub to discover, fork, and contribute to over 100 million projects. Mar/2020 Colorizing. Robotics:AI Techniques . Machine Learning . Computational photography combines ideas in computer vision, computer graphics, and image processing to overcome limitations in image quality such as resolution, dynamic range, and defocus/motion blur. Application Integration . Office Hours: Thursday 3-5PM - write an email to oliver.cossairt (a) northwestern.edu to book a 10min slot. I am interested in Image Processing, Computational Photography and Computer Vision. Since WebRTC is used for capture and transport, users have to rely on other resources to complete their application, such as a dedicated server to handle image and data processing tasks and return useful results. Announcements and discussions will take place on CampusWire. This course is the first in a two-part series that explores the emerging new field of Computational Photography. Much of my research is about Deep Learning and Camera Pipeline. I did my bachelors at Nanjing Agricultural University. However, utilising a technique from computational photography called coded aperture, we can obtain absolute depth using just a single image.The idea in coded aperture is similar … We will provide a Nvidia Tegra tablet for each student in the course. (5) Northwestern Neuroimaging and Applied Computational Anatomy (Lei Wang) 20 min (6) Michigan Institute for Data Science (Ivo Dinov), 20 min 12:15‐1:15 Lunch Break 1:15‐3:15 Unconference Breakouts Informal self‐organized sessions (30‐minutes each), round‐robin rotations. EECS 211 and/or 230 or permission from instructor. subject to change as the course progresses. I am an Assistant Professor in the EECS Department at Northwestern University. These glass tiles were part of a sample set from the Kokomo Opalescent Glass Works in Indiana, famous for having supplied glass to Louis Comfort Tiffany. : July/2020: Starting in September I will be joining Dr. Vladlen Koltun's Intelligent Systems Lab at Intel as Research Scientist resident. Here's my GitHub. Face Morphing. Aug/2020: One paper accepted at SIBGRAPI 2020! Unsupervised Deep Learning for Computational Photography and Imaging Self2Self: Self-Supervised Image Denoising Self2Self with dropout: Learning self-supervised denoising from … New methods offer unbounded dynamic range and variable focus, lighting, viewpoint, resolution and depth of … Oliver Cossairt I did not provide the MATLAB scripts for these projects publicly (github, etc.) Florian’s application uses webrtc-perception to access the front-facing camera on a device and change camera settings for the connected client. Image Classification. Office Hours: Thursday 3-5PM - write an email to florian.willomitzer@northwestern.edu to book a 10min slot. Computational photography combines ideas in computer vision, computer graphics, and image processing to overcome limitations in image quality such as resolution, dynamic range, and defocus/motion blur. Furthermore, my system needed to work without requiring my colleagues to possess special hardware or be familiar with the nuances of browser APIs or web development. Late Policy: If EITHER there is nothing on Canvas OR your code has not been pushed to by 11:59pm on the due date, you fail the assignment. thesis! Also, put up a “safety” submission on Canvas with what you currently have, an hour prior to the deadline. Students will write programs that run on the phone to capture photos. The client signals to the server when it is ready to begin data capture, and the server responds with a signal to start “measuring” with the device. 2. cs1290tas@lists.brown.edu—your s… Soham Ghormade. Implementing PMD techniques on consumer devices using webrtc-perception is an alternate way to measure the surface shape by instead “scanning” the glass with the mobile device. ELEC_ENG 395, 495: Computational Photography Seminar “guest lecturer“, Northwestern University, 2020. CS331 lecture: All lectures will held live on zoom and linked through canvas. This is a prediction of what will be covered in each week but the schedule is Jeremy Lainé has put together a very useful package and I highly recommend giving it a closer look. If you have a question about whether something may be considered cheating, ask, prior to submitting your work. My Ph.D. thesis was closely related to the tasks which involve moving objects present in videos or images captured from different view-points. Canny Edge Detection I am currently a third-year master student at Beihang University, where I work on computational photography under the supervision of Prof. Feng Lu. Finally, there are some details below the webrtc-perception metapackage description that talks about some specific applications for this technology, both of which have unique implications for scientific study of artistic works. Computational Photography & Computer Vision Image Warping and Mosaicing. https://www.sciencemag.org/news/2019/02/new-app-reveals-hidden-landscapes-within-georgia-o-keeffe-s-paintings, https://www.mccormick.northwestern.edu/news/articles/2019/02/diagnosing-art-acne-in-georgia-okeeffe-paintings.html, featured in Optics Express Vol. See CANVAS for the link to invite your to create your Github repository for the assignments. Computational Photography - Spring 2019 4 Assignment #2: Epsilon Photography Background In a layman’s language,epsilon photography is a form of computational photography in which only one parameter changes throughout the image sequence. This course will consist of six homework assignments and no midterm or final exam.
Straight Rope Png, Stylish Shoes Png, Computer Technician Duties And Responsibilities, Marucci Oxbow Series, Kinder Schoko-bons Uk, Emergence Of Global Economy,