The woman told police she was in a conference room at the main Old Post Road library around 3:15 p.m when a man sitting nearby pulled out his penis and began rubbing it and looking at her.
She tried to take a photo with her cell phone, but the man covered up, told her he was sorry and exited the building, police said.
Library staff told police the man did not look familiar.
Click here to follow Daily Voice Fairfield and receive free news updates.