Here's the relevant part of the help:
The division operations have an optional maximum gain parameter parameter which limits the boost which occurs when the divisor becomes very small (or zero), such as where the divisor has notches in its response, and so produces a more stable and manageable result. Division may also be constrained to operate within a desired frequency span, with an upper and/or lower frequency limit. Outside those limits the behaviour depends on whether gain limiting is selected. If the gain limit is being used REW treats the operation as an attempt to produce a band limited correction filter and division reverts to unity gain, blended over a 1 octave span centred on the limit frequency. If the gain limit is not being used division reverts to the numerator, offset to align with the result within the band limit.