2.0.12.alpha, patch 1(glcm)

[Karim BELABAS <Karim.Belabas@math.u-psud.fr>]

This patch corrects a bug reported (directly to us) by Thomas Papanikolaou in
the glcm function, which could occur (rarely) when the operands had both more
than 150 digits, and became systematic after ~5000 digits. Symptom: wrong
result due to stack corruption (not even guaranteed to be a multiple of
either operand).

Many thanks to Gerhard Niklasch for a careful analysis of the problem.

  Karim.

*** src/basemath/polarit2.c.orig        Fri Nov  6 16:08:22 1998
--- src/basemath/polarit2.c     Fri Nov 13 13:13:00 1998
***************
*** 1288,1301 ****
    if (gcmp0(x)) return gzero;
    if (tx==t_INT && ty==t_INT)
    {
!     (void)new_chunk(lgefint(x)+lgefint(y));
!     p1=mppgcd(x,y); p2=divii(y,p1); avma = av;
!     /* should be diviiexact. But still buggy...*/
!     p2 = mulii(p2,x); if (signe(p2)<0) setsigne(p2,1);
!     return p2;
    }
!   p1=ggcd(x,y); p2=gdiv(y,p1);
!   p2=gmul(p2,x);
    if (typ(p2)==t_INT && signe(p2)<0) setsigne(p2,1);
    if (typ(p2)==t_POL)
    {
--- 1288,1300 ----
    if (gcmp0(x)) return gzero;
    if (tx==t_INT && ty==t_INT)
    {
!     p1 = mppgcd(x,y); if (is_pm1(p1)) { avma = av; return mulii(x,y); }
!     p2 = mulii(divii(y,p1), x); 
!     if (signe(p2)<0) setsigne(p2,1);
!     return gerepileupto(av, p2);
    }
!   p1 = ggcd(x,y); if (gcmp1(p1)) { avma = av; return gmul(x,y); }
!   p2 = gmul(gdiv(y,p1), x);
    if (typ(p2)==t_INT && signe(p2)<0) setsigne(p2,1);
    if (typ(p2)==t_POL)
    {

--
Karim Belabas                    email: Karim.Belabas@math.u-psud.fr
Dep. de Mathematiques, Bat. 425
Universite Paris-Sud             Tel: (00 33) 1 69 15 57 48
F-91405 Orsay (France)           Fax: (00 33) 1 69 15 60 19
--
PARI/GP Home Page: http://pari.home.ml.org