Ben Diven was born in northern California, and graduated from University of California, Berkeley with a degree in physics in 1941. In 1942, he approached Robert Oppenheimer for a job, and was brought on to the early Manhattan Project. In mid-March of 1943, Diven went with Oppenheimer to Los Alamos. He oversaw much of the construction at Los Alamos, regularly communicating with Oppenheimer about vital changes. He also was in charge of distributing equipment as it was shipped into Los Alamos. He then went to work under Bruno Rossi, doing critical mass testing.
He was married to Becky Diven, who also worked on the Manhattan Project at Los Alamos.