Do I need to build one airship dock for each player?

No. Players can share the same airship dock.

However, only one player can dock at the same time. You can also land your airship without a dock anywhere that has enough space.