[Sumover-dev] [svn commit] r4139 - vic/branches/mpeg4/codec/p64

[sumover-dev at cs.ucl.ac.uk]

Author: douglask
Date: Tue Apr  8 05:02:37 2008
New Revision: 4139

Modified:
   vic/branches/mpeg4/codec/p64/p64.cpp

Log:
Local variable sx of type u_int sometimes had a value of 0xfffffffe (i.e -2),
which would screw up the 64bit pointer arithmatic and subsequently cause
an access not within mapped region segmentation fault on x86-64 linux.


Modified: vic/branches/mpeg4/codec/p64/p64.cpp
==============================================================================
--- vic/branches/mpeg4/codec/p64/p64.cpp	(original)
+++ vic/branches/mpeg4/codec/p64/p64.cpp	Tue Apr  8 05:02:37 2008
@@ -140,9 +140,12 @@
 	miny_ = height_;
 	maxx_ = 0;
 	maxy_ = 0;
-	marks_ = 0;
 
 	allocate();
+
+	// invalidate the just-changed-block table (marks_ buffer)
+	// to avoid buffer overruns
+	marks_ = 0;
 }
 
 #if BYTE_ORDER == LITTLE_ENDIAN
@@ -947,9 +950,9 @@
 			mvblka(in, out, stride);
 		return;
 	}
-	u_int sx = x + (mvdh_ / sf);
-	u_int sy = y + (mvdv_ / sf);
-	u_char* in = back + sy * stride + sx;
+	int sx = x + (mvdh_ / sf);
+	int sy = y + (mvdv_ / sf);
+	u_char* in = (u_char*)((intptr_t)back + sy * stride + sx);
 	if (mt_ & MT_FILTER) {
 		filter(in, out, stride);
 		if (tc != 0) {