I see no reason why reactions to an attack cannot be used outside of an initiative order.
They can, but a surprised creature cant take reactions until after it has finished its first turn in combat (see the PHB).
Example:
An Orc is creeping down a cluttered warehouse flitting from from crate to crate (Stealth check result 17), and spots two PCs (Passive perception 10) 30' away. The PCs fail to notice the Orc as its Stealth check > their Passive perception.
The DM decides the Orc lifts his crossbow to his eye, aims at the 1st PC and pulls the trigger. He states to the players:
'Suddenly from out of the darkness near the crates to the South, you hear the twang and whistle of a crossbow bolt being launched!
Roll initiative.'
The DM now resolves that action.
He rolls initiative to see if the PCs can react in time to the Orcs action.
Lets have two examples (one where the Orc goes first, and one where it goes last):
Example 1:
Initiative: Orc rolls 15. The PCs roll 10 and 5. Orc goes first.
Round 1:
The Orc goes first. Its attack is resolved against PC 1 (at
advantage thanks to the Orc being hidden at the time the attack was made). It gets +6 to hit, and rolls a 3 and a 5 on its attack roll, missing the first PC (who has an AC of 18). Note that neither PC can take a reaction to this attack, as they havent had their first turn in combat yet.
PC 1 goes next on initiative count 10. He stares blankly at the crossbow bolt lodged in his shield mouth agape (he is surprised so is unable to take any actions, or move on his turn). His turn then ends, and he can now take reactions.
PC 2 goes next. He is also surprised and cant do anything on his turn. His turn then ends, and he can now take reactions.
Round 2:
The Orc loads his crossbow, and again takes a shot. He is no longer hidden (he revealed himself last round when he made at attack, and making an attack reveals your position after resolving that attack, hit or miss) so there is no advantage on this attack. He shoots at PC 2 (who isnt wearing any armor) as he looks the easier target. He rolls a 12 for a total attack roll of 18 vs PC 2's AC of 15 - a hit!
PC 2 is a Wizard with the Shield spell. He uses his reaction to cast it (crossing off the spell slot) and raises his AC to 20. The Orcs bolt is deflected by the magical barrier.
Next comes PC 1, who can finally take an action. He draws his sword, screams 'STABFELLOW JENKINS CHAAAAAAARGE! and moves 30' to the Orc and makes a melee attack...
And so on.
Example 2:
Initiative: Orc rolls 5. The PCs roll 20 and 15. Orc goes last.
Round 1:
PC 1 goes first on initiative count 20. He fumbles for his sword, mouth agape (he is surprised so is unable to take any actions, or move on his turn). His turn then ends, and he can now take reactions.
PC 2 goes next. He is also surprised and cant do anything on his turn. His turn then ends, and he can now take reactions.
The Orc goes next. Its attack is resolved against PC 1 (at
advantage thanks to the Orc being hidden at the time the attack was made). It gets +6 to hit, and rolls a 15 and a 5 on its attack roll, hitting the first PC (who has an AC of 18) with a 21, dealing 10 points of piercing damage.
Unbeknownst to the Orc, PC 1 (Our hero, Stabfellow Jenkins) is a Warlock. He uses his reaction to cast
Hellish Rebuke, targetting the Orc and dealing 22 points of fire damage to it. Note that this PC could take a reaction to this attack only as they have had their first turn in combat already (even though they could take no actions during that turn as the were surprised).
Round 2:
PC 1 goes first thos round and can finally take an action. He finally manages to draw his sword, screams 'STABFELLOW JENKINS CHAAAAAAARGE! and moves 30' to the Orc and makes a melee attack...
And so on.