New exhibition in Venice puts challenging work that risked offending “stuffy visitors” in context