A separation oracle for a convex space S is a polynomial-time algorithm A that, given a vector x, returns "yes, x is in S" if x∈ S or, otherwise, returns "no, x is no in S" along with a separating hyperplane.
A separating hyperplane is a hyperplane, defined by some vector v, such that v.x < v.y for all y ∈ S.
Generally, given a separation oracle, the ellipsoid method returns, in polynomial time, either a proof that the corresponding space is empty, or a point that lies within the space. By adding additional constraints, this suffices to optimize a linear function over the space.
More precisely, one must provide the ellipsoid method with an initial bounding region B such that B contains S. The ellipsoid method returns either a point in S, or a proof that volume(S) ≤ volume(B) / exp(nO(1)). Typically the latter is enough to conclude that S is empty.
In fact, an exact separation oracle is not required, it suffices to have a so-called weak separation oracle, one that separates the query vector x approximately from S.
Here we are skipping over details about the model of computation, including how to compute with numbers and vectors in polynomial time, when the number of bits needed to represent the numbers is an issue.