Goals in each half from the relegation threatened visitors ensured they took the three vital points at Recreation Park.
Alfie Brunskill opened the scoring 24 minutes into the game as Ryhope went into the break a goal down.
The visitors doubled their lead on 68 minutes as Jamie Armstrong secured the victory for Penrith.
Ryhope headed into the match in fine form unbeaten in five in all competitions, with the visitors winning only two of their last five.
Penrith went into the game on the back of an impressive 2-1 victory over league leaders Consett, while Ryhope won 4-1 away at Whitley Bay last time out.
The home side started brightly with defender Danni Lay striking the post in the 10th minute as he looked to put his side into the lead.
Colliery Welfare continued their pursuit of an opener when Robbie Bird forced a good save from Aaran Taylor having burst into the box.
In the 24th minute it was the away side who found the breakthrough as Brunskill capitalised on a communication error between the Ryhope goalkeeper and the defender to poke the ball under Jordan Harkess in the Ryhope goal.
Ryhope searched for an equaliser when just before half-time the ball was floated into Bird in the box whose speculative effort went wide of the goal.
Brunskill almost got his second of the game in the 62nd minute as he latched onto a loose ball in the box only to see his close-range effort saved by Harkess.
Penrith got their second of the game on 68 minutes as Armstrong latched onto a loose ball in the Ryhope half and sent his shot from the edge of the box into the bottom right corner of the Ryhope goal.
Ryhope looked to get a goal back as Bird saw an effort go over the bar in the 84th minute.
Colliery Welfare will be hoping to bounce back from the defeat when they face Newcastle Benfield at home on Saturday.