Signs don't vote, people do. I remember on election day last year being at the polling location at the church on North A Street. I was waving signs and chatting with people as they were coming from their cars. In the line-up of other people holding signs on the curb outside the poll, there was a man with a Golden/McNamara combination sign. We started chatting about the day and he said that he knew that two people on his street were voting for me - he and his wife! He didn't go on to explain the apparent disconnect between the signs that he was holding and their household's vote, but it proved my point that signs don't vote and they can be misleading.
There are many businesses, local business owners that happen to live in town or residents that don't want to indicate who they are voting for, especially when an incumbent is in the race. There are many quiet supporters out there that think, rightly or wrongly, that their business will be hurt or dealings with the city will be affected if they publicly display their preference. There are also households with multiple voters that may not be voting all the same way - another deceiving aspect of political signs.
Signs can also be a give away as to who your supporters are and they become targets of your opponent. I had reports of my opponent coming by three and four times to some houses with my sign in front - to the point that the residents told her to stop coming by that they weren't changing their vote. So, if you have a sign in your yard, be prepared to continually re-affirm your commitment to that candidate.
I also have it from a good source that a certain "side" targets the area near polling locations 72 hours before the election and selectively removes certain signs from the area.