How much will replacing brick with siding affect house value?

We are about to have some foundation problems fixed. There is major rotting around some of the windows, especially where the brick has separated, plus major cracking in the bricks. The house is 1890 square ft and was built in 1978. We are considering taking off the brick and installing vinyl siding when we replace the windows. How much will that change the value of our house?

The brick is buff colored, not really pretty, and there are areas where it's a different color (specifically where we tore down an old chimney, put in a new woodstove, and ran the woodstove chimney through the roof.) There are places where they got mortar on the brick and just left it, and places where the former owner oversprayed the trim and got it on the brick.

Siding would definitely make it more attractive, but we are concerned about how much value we would lose as opposed to having some of the brick redone, especially since it may have to be taken off in places anyway if water has leaked at the windows.