Doesn't the target have to be right inside the twisting colours for HP to work?
The spell does not actually say this as it does not specifiy the dimensions of the colors only the area of the creatures that can see it. Moreover colors are not physical, so even if this is the case, the cover only effects the point you target. You would have to to target outside the wall or it the target would become the edge of the wall if you tried to centerthe spell inside it. However the wall would not stop the twisting pattern of colors from going through it, just it doesn't stop the light from a torch.