A Second Officer was third in command of a vessel or ship, and normally a department head as well.
Some starships, typically those on deep space missions, kept the position separate from other duties.
Montgomery Scott - USS Enterprise and USS Enterprise-A
Data - USS Enterprise-D and USS Enterprise-E
Jadzia Dax - Deep Space Nine (2369 - 2374) and USS Defiant (2372 - 2374)