I came across this the other day and had a bit of trouble finding out what it meant. Luckily, I managed to get an explanation from Bo Jeanes. It turns out that '~>' means 'up to the next major version'.
So, for example, 'some-gem' depends on 'other-gem ~> 1.4'. Now, if I install version 1.5 of 'other-gem' and cleanup version 1.4 the dependency is still met because 1.5 is still within the same major version.
However, let's say version 2.0 came out and I installed it and removed version 1.5. The dependency is no longer met due to 2.0 being the next major version. Understand? Good.
Assuming that gem maintainers are smart about what they deem as major updates I think setting up '~>' dependencies is a good idea.