The uncompromising artist turn down many exhibition offers but never veered from his vision for a work