
One man has a bare chest and one man wears a swimsuit with a top in this 1937 illustration from Woman’s Home Companion. July 1937, p. 74.
Nude bathing for men was an accepted tradition in Victorian times. (A stretch of river called Parson’s Pleasure was reserved for this purpose at Oxford University until 1991.) But as “mixed” bathing became popular near the end of the 19th century, both men and women were expected to cover up from breastbone to knee.

Man’s bathing suit from Sears catalog, Spring 1910. Sleeveless swimming suits for men were also for sale.
1920’s bathing suits were clinging, but very similar for both sexes.

Bathing suits from the Sears catalog, Spring 1925. The swim suit worn by the seated man is not very different from the woman’s suit.
Practices varied from place to place but, at public beaches and pools in the U.S., men were usually required to wear suits that covered their nipples until the mid-nineteen thirties.

Men’s swimming suits from Sears, Spring 1935. Left, an elasticized “Speed Suit” suspended from the shoulders. Center, trunks with a separate tuck-in shirt. Right, a “two-purpose suit” whose top attaches with a zipper.
The “Speed Suit” (left) has attached trunks and “elastic-ribbed fabric.” The “High Waisted Trunks” at center are shown with a separate all-wool shirt which tucks into the suit at front and back. The “two-purpose” Zip Top Suit” at right has a zipper in front that allows you to remove the “shirt” part.
By 1934, it was becoming acceptable for men to swim bare-chested, but rules for public and private beaches and pools differed, so bringing an optional top would save embarrassment. (Speaking of embarrassment, I wonder: when the trunks were not suspended from the shoulders, was a belt necessary to support the weight of water-logged wool knit trunks?)
This vintage suit, from Macy’s, has a similar zipper front and a rather bare X back:

Man’s swim suit from Macy’s, circa 1930s; the detachable top connects to the trunks with a large metal zipper.
This illustration from Womans’ Home Companion, 1937, shows that some men — in this case, two out of three — continued to wear the top even when not required to do so.

Men’s bathing suits with tops, WHC, February 1936 illustration.
The older man is wearing a more conservative, covered-up swimsuit.
According to Esquire magazine in 1934,

Esquire, July 1934, page 118.
This implies that shirtless swimming was permitted on some public beaches in 1933, and earlier [1932] at some private beaches and pools.

Esquire, July 1934, p. 118. Men’s swimming trunks without chest coverage. The punning caption read: “Even the Public Beaches Embrace the Nude Deal.”
The man at left is wearing a shirt tucked into his trunks.

Esquire, July 1934, p. 118.
In the same July 1934 issue, this ad for Mansco Sportswear shows several conservative looks:

Ad for Manhattan Mansco sportswear and swimming trunks. Esquire, July 1934.
However, this ad from Gantner and Mattern Co. shows much tighter-fitting trunks — and no top.

Ad for Gantner “Wikies” swim trunks, Esquire, July 1934.

Gantner “Wikies” man’s swim trunks. Ad, Esquire, July 1934. A “Snapper Shirt” top for Wikies was available separately, presumably to snap on at beaches where swimming with a bare chest was still not permitted.
The Wikies’ high waist reflects the high-waisted men’s trousers then in fashion. Wikies’ snug fit was probably possible because of the recent [1931] invention of Lastex yarn, which even appeared in men’s suit fabric in 1934 ads.

From a Lastex ad, Esquire, March 1934, p. 8. “Lastex, the spun elastic yarn, is now weaving comfort into everything a man wears — into his business suit, Tuxedo, sportswear, bathing suit, riding clothes, shirt, …underwear, pyjamas….”
The Lastex company ran a series of advertisements in Esquire magazine showing men’s suits, tuxedos, etc. which were made with stretch fabrics — in 1934!

Beach and resort wear, including “pretty snug” men’s swimming trunks, worn bare-chested. Esquire, August, 1934, p. 133. L. Fellows, illustrator.
This editorial illustration appeared in a women’s magazine in 1935:

Illustration by Warren Baumgartner, Woman’s Home Companion, May, 1935.
Perhaps the acceptance of bare chests had something to do with Hawaii:

A Hawaiian surfer in a Dole Pineapple ad, May 1934. Delineator.
I can’t help noticing that Esquire chose to use men “of a certain age” to model swimsuits in its editorial fashion articles. The women’s magazines, however, pictured younger, athletic-looking men wearing swimsuits in their illustrations, just as Esquire favored voluptuous women in its cartoons….