The Colts are really taking a risk leaving the 3rd pick sitting out there like that.
Ballard's a Ravens style, conservative, maximize your picks style of GM. If it was up to him, I'm sure he'd rather sit at 4 and let the draft come to him. However they've got a massive QB need and lots of other QB needy teams sitting behind them. They've got the advantage that Arizona can still get Will Anderson, so they could move up relatively cheaply, yet they're leaving it open for anyone to jump past them.
Even if Irsay has his sights set on Lamar, you'd think they'd still want the fallback option of being able to choose their QB at 3.