The Walnut Street Causeway Bridge is part of the former Hudson Toll Bridge, now fondly recalled as the Old Toll Bridge. The river crossing consisted of this smaller bridge on the Wisconsin side of the crossing, a half-mile long earthen causeway, and then the main channel bridge. The main channel was crossed by a larger steel truss.

The original bridge at this location was built in 1913. It was worn out and narrow, so it was replaced with a more modern bridge in 1940. The 1940 bridge was retired in 1951 when a new US-12 bridge was built about a half-mile downstream, and the Toll Bridge was closed. While the main channel span was quickly removed and recycled, the causeway bridge remained in place.

Today, the causeway bridge allows for pedestrians and bicycles to use the causeway. Once can walk or bike out to the very end, which was the location of the toll booth. It is a nice spot to watch boat traffic, and there is a small sandy beach. The other use forthe causeway bridge is to support tour boats. The tour boats dock along the causeway, and passengers walk a short distance from shore to board the vessles. Vehicles in support of the boats, such as those bringing out food, ice, and mechanical parts, are also allowed to use the causeway.