Beguiled by a Baron by Christi Caldwell

A Lady with a Secret... Partially deaf, with a birthmark marring her face, Bridget Hamilton is content with her life, even if she's been cast out of her family. But her peaceful existence—expanding her mind with her study of rare books—is threatened with an ultimatum from her evil brother—steal a valuable book or give up her son. Bridget has no choice; her son is her world.
A Lord with a Purpose... Vail Basingstoke, Baron Chilton is known throughout London as the Bastard Baron. After battling at Waterloo, he establishes himself as the foremost dealer in rare books and builds a fortune, determined to never be like the self-serving duke who sired him. He devotes his life to growing his fortune to care for his illegitimate siblings, also fathered by the duke. The chance to sell a highly coveted book for a financial windfall is his only thought.
Two Paths Collide... When Bridget masquerades as the baron's newest housekeeper, he's hopelessly intrigued by her quick wit and her skill with antique tomes. Wary from having his heart broken in the past, it should be easy enough to keep Bridget at arm's length, yet desire for her dogs his steps. As they spend time in each other's company, understanding for life grows as does love, but when Bridget's integrity is called into question, Vail's world is shattered—as is his heart again. Now Bridget and Vail will have to overcome the horrendous secrets and lies between them to grasp a love—and life—together.