On several walls in my home it looks as if someone threw water on them and let it run down the wall. The stains are underneath windows mainly and one is on the other side of the bathroom. What causes this?

Hi - I'm hoping someone can help me figure out what one of my walls is made of. I live in a fairly old house that has mostly plaster walls. There are two walls in my living room, however, that have a much harder material than just wood under the plaster. If I use a wood drill bit, I can only get through about 1/8" of plaster before it stops. With a concrete bit, I get a lot further. The first dust I see coming out the drill hole is white (the plaster), then grayish (concrete?) and then, deeper in is a light brown dust. Behind the brown dust the wall is hollow as far as I can tell. This wall has an archway dividing my living room. It also might be supporting the exterior wall on the 2nd floor. Any ideas what the brownish material might be? Is it safe to be drilling into it? As far I can tell there are no pipes or wiring going through this wall. Thanks!