In this solution I have included a user input to adjust the number of bars to include in the count. At the bottom of the code I included counts for both the overbought as well as the oversold. Only one of these can be plotted at one time in the watchlist column. You can adjust which one is displayed using the last line in the code.
declare lower;
input spanOfBars = 100;
input rsiLength = 14;
input rsiPrice = close;
input rsiAverageType = AverageType.WILDERS;
input overbought = 60;
input oversold = 40;
def netChangeAverage = MovingAverage(rsiAverageType, rsiPrice - rsiPrice[1], rsiLength);
def totalChangeAverage = MovingAverage(rsiAverageType, AbsValue(rsiPrice - rsiPrice[1]), rsiLength);
def changeRatio = if totalChangeAverage != 0 then netChangeAverage / totalChangeAverage else 0;
def rsi = 50 * (changeRatio + 1);
def countOversold = Sum(rsi > overbought, spanOfBars);
def countOverbought = Sum(rsi < oversold, spanOfBars);
plot data = countOversold;