[ipoib] Remove the queue set abstraction