gallium/util: fix returning empty box for rectangle intersection
authorRoland Scheidegger <sroland@vmware.com>
Tue, 1 Sep 2015 22:29:17 +0000 (16:29 -0600)
committerBrian Paul <brianp@vmware.com>
Tue, 1 Sep 2015 22:29:17 +0000 (16:29 -0600)
These functions deal with inclusive coordinates, hence a 0/0/0/0 rect
returned when there's no intersection doesn't actually represent an empty
rectangle. Hence return 0/-1/0/-1 instead.
This fixes some problems in llvmpipe with empty scissor rects (which up
to now didn't really matter because while the intersect test returned the
wrong result all pixels were scissored away later anyway).

src/gallium/auxiliary/util/u_rect.h

index dea1e1ecff38406b188bb4da944c1852ee5fca3e..221d9188730eaa98af9ac7d4b53f6212578c4b7b 100644 (file)
@@ -87,7 +87,12 @@ u_rect_possible_intersection(const struct u_rect *a,
       u_rect_find_intersection(a,b);
    }
    else {
-      b->x0 = b->x1 = b->y0 = b->y1 = 0;
+      /*
+       * Note the u_rect_xx tests deal with inclusive coordinates
+       * hence all-zero would not be an empty box.
+       */
+      b->x0 = b->y0 = 0;
+      b->x1 = b->y1 = -1;
    }
 }