[java]Question with the BigInteger class - Tilted Forum Project Discussion Community

wombatman · 03-12-2005, 05:34 PM

I'm trying to factor a large (193 digit) number into 2 prime factors. And I'd like to try to use the sieve method to speed this up. Unfortunately, I can't initialize a BigInteger array to a large enough size to do this. Can anyone help me come up with a way around this? As it is, I'm just looping through, adding/subtracting 2 every time, checking whether the new number is prime, and seeing if it divides evenly into the main number (the 193 digit one).

Thanks a ton.

03-14-2005, 12:48 PM

Do we get a share of the prize? Factoring a 193 digit number into 2 prime factors could earn you something in the region of $20,000! It's not supposed to be easy - why not initialise your BigInteger array in smaller, bite-size chunks?

If you could do that, you could farm off the processing of each chunk to a different processor, and, given enough processors, find the results you're looking for in less than a year.

I hope I've not jumped to any conclusions here, but please explain further in case I've got the wrong idea.

03-14-2005, 12:52 PM

In case no-one knows what I'm on about, here's the link to the remaining RSA prize numbers. There's something like $600,000 in unclaimed prize money left to win...

http://www.rsasecurity.com/rsalabs/node.asp?id=2093

$20,000 for a 193 digit number.

03-14-2005, 12:54 PM

In fact, this could form the basis of a separate thread - we could try and put together a join TFP Prime-Factor challenge team - work out some decent code, pool some resources and win some cash?! Any interest guys?

MageB420666 · 03-14-2005, 01:46 PM

I take it any funds(reward money) recieved would go to the TFP?

03-14-2005, 01:58 PM

Either directly to the TFP, or towards funding a TFP get-together for all those involved. The PR alone should be enough to generate contributions.

03-14-2005, 01:59 PM

However, it's a more than just a tricky job to actually factor the numbers, I wouldn't start booking any flights just yet...

wombatman · 03-14-2005, 05:29 PM

What zen_tom said. I've had 5 computers running pretty continuously for at least 2 weeks doing numbers. And Zen, I had just thought about doing the smaller arrays. I tried doing a 2-dimensional array and quickly found out that I do not have that much memory.

And yeah, this is for the RSA thing. I honestly did not mean to mislead if anyone didn't know where this was coming from.

One potential problem I can see with trying to a joint venture on this would coordinating who is responsible for what range of numbers and so on. Different speed and different types of processors will of course run at different speeds. There is also a somewhat noticeable difference in the time to check the 90+ digit numbers (I cut the 193 in half and am focusing on the lower half, since there must be one factor at or below 97 digits) and the 11 and 12 digit numbers (starting from say 3). For the time being, I'll continue working on it solo I guess. It should be noted, however, that I was giving careful consideration to a donation of some of the money to TFP as I came across the Erathosthenes(sp?) Sieve method here. Now, due to the size of the numbers involved, I'm having to rethink the details of the method, but I think a thousand or two thousand dollars wouldn't be out of the question, and I imagine that would help a good deal with bandwidth/server costs.

Thanks again for the comments, Zen.

Rangsk · 03-14-2005, 08:42 PM

Why Java? You realize that anything in Java will be 10 times slower than anything in C++, which is a good 33% slower than anything in Fortran.

wombatman · 03-14-2005, 09:21 PM

Well, mostly because Java is the class I'm in. I have a basic understanding of C++, but don't have a compiler for it currently and also don't quite know how to go about working with really large numbers (read 193 digit+). If someone could direct me somewhere that I could learn about handling a number that large, I would certainly give it a shot, especially since I would like to speed this up a bit.

wombatman · 03-14-2005, 09:37 PM

Alright, got a free compiler and downloaded the GMP library (supposed to be able to handle massive numbers), but I have no idea how to use GMP.

Edit: Tried using long double (largest capacity variable I could find for c++), but it output a decimal with an e192, so that's no help. And I tried unsigned long double, but that produced an error saying that I can't use short, signed, or unsigned before long double. Still looking for any assistance with this that anyone is willing to provide.

03-15-2005, 04:43 AM

You might have to manufacture your own class of number, HugeInteger or something - not sure how I'd go about this, but it might be stored as an array of chars. Floating point numbers like floats and doubles can be innaccurate when they either have to store very large or very small numbers in a precise manner - which wont help in this problem.

You were right first time with the BigInteger. Allocating a large array got a near 600 bit record size is of course going to simply going to up a fair amount of memory.

The alternative is to store the numbers in memory using some constructed class that you've designed to optimise the processes you are running. For example, their might be some useful heuristics that will help strip out various factors, or maybe converting the number into decimal, you could run some kind of mask/pattern matching on it (I'm not 100% sure what I'm talking about here actually so I'll shut up quickly before I embarrass myself) Anyway, good luck and I'll have a think about how else we could approach this.

vinaur · 03-15-2005, 06:02 AM

So are you going to eventually go through a bunch of primes and try to divide the given number by those primes???

Wombatman: you said that you split up the number into two parts and are working with the lower part. Well, when you do find the number that is a factor of the first half, it does not necessarily mean that it will also be a factor of the whole thing. You would have to test the factor against the whole number to determine whether it really is a factor. I was thinking more of the other way around. You look at the higher part first and when you find a factor, you check it against the lower part.

wombatman · 03-15-2005, 09:33 AM

Sorry, I meant that I am only looking for factors of the target number (the big 193 digit one) in numbers that are 97 digits long or less. Every time I find a prime I am dividing it into the main number (the big one) and seeing if it divides evenly. When I find the prime that does, I'm having the program write the other factor (whatever the big number divided by the prime is) into a text file and stopping. This way I have both factors.

Rangsk · 03-17-2005, 08:29 PM

I'd like to apologize, I recently did some searching and it turns out that in a lot of situations nowadays, Java is actually FASTER than C++ because of Just in Time compiling and Hotspot, etc. Java is no longer interpreted, instead the bytecode is compiled just before running - pretty cool if you ask me

wombatman · 03-30-2005, 10:50 AM

One more question since I'm still running this program...does displaying the prime numbers I find slow down the process significantly? And if so, would writing them to a file make a difference? The only reason I'm printing them all out is so I can stop the program at any time to free up CPU power.

03-30-2005, 11:07 AM

i/o of any kind (network, file - or even to the screen) will slow down the process - I'd try to cut it down to a minimum where possible.

a-j · 03-30-2005, 11:14 AM

Console I/O can slow down a program, but I don't think it would make a huge difference if you are only outputting a number every so often, if the console is scrolling like crazy, then it is probably being inhibited. If the number is being output in hex/binary I would think it is resonably fast, if it is doing it in base 10, then it would probably be slower.

wombatman · 03-30-2005, 12:37 PM

Ok, well I'm printing out every prime the program finds (not prime factors since they're what I'm actually trying to discover), so I guess I should try not outputting them to the screen and maybe writing every 500th prime or so to a file so if something does happen, I'm not too far behind.

And if it matters, I'm printing out a string to the console since that's how the BigInteger class seems to handle the long numbers.

03-12-2005, 05:34 PM	#1 (permalink)
wombatman Insane	[java]Question with the BigInteger class I'm trying to factor a large (193 digit) number into 2 prime factors. And I'd like to try to use the sieve method to speed this up. Unfortunately, I can't initialize a BigInteger array to a large enough size to do this. Can anyone help me come up with a way around this? As it is, I'm just looping through, adding/subtracting 2 every time, checking whether the new number is prime, and seeing if it divides evenly into the main number (the 193 digit one). Thanks a ton.

03-14-2005, 12:48 PM	#2 (permalink)
zen_tom Guest	Do we get a share of the prize? Factoring a 193 digit number into 2 prime factors could earn you something in the region of $20,000! It's not supposed to be easy - why not initialise your BigInteger array in smaller, bite-size chunks? If you could do that, you could farm off the processing of each chunk to a different processor, and, given enough processors, find the results you're looking for in less than a year. I hope I've not jumped to any conclusions here, but please explain further in case I've got the wrong idea. Last edited by zen_tom; 03-14-2005 at 01:00 PM.. Reason: $20,000 not $250,000 - though, if you got a good thing going on, you could win all-sorts

03-14-2005, 01:46 PM	#5 (permalink)
MageB420666 <Insert wise statement here> Location: Hell if I know	I take it any funds(reward money) recieved would go to the TFP? __________________ Apathy: The best outlook this side of I don't give a damn.

03-14-2005, 08:42 PM	#9 (permalink)
Rangsk Crazy Location: San Diego, CA	Why Java? You realize that anything in Java will be 10 times slower than anything in C++, which is a good 33% slower than anything in Fortran. __________________ "Don't believe everything you read on the internet. Except this. Well, including this, I suppose." -- Douglas Adams

03-14-2005, 09:37 PM	#11 (permalink)
wombatman Insane	Alright, got a free compiler and downloaded the GMP library (supposed to be able to handle massive numbers), but I have no idea how to use GMP. Edit: Tried using long double (largest capacity variable I could find for c++), but it output a decimal with an e192, so that's no help. And I tried unsigned long double, but that produced an error saying that I can't use short, signed, or unsigned before long double. Still looking for any assistance with this that anyone is willing to provide. Last edited by wombatman; 03-14-2005 at 10:02 PM..

03-14-2005, 12:52 PM	#3 (permalink)
zen_tom Guest	In case no-one knows what I'm on about, here's the link to the remaining RSA prize numbers. There's something like $600,000 in unclaimed prize money left to win... http://www.rsasecurity.com/rsalabs/node.asp?id=2093 $20,000 for a 193 digit number.

03-14-2005, 12:54 PM	#4 (permalink)
zen_tom Guest	In fact, this could form the basis of a separate thread - we could try and put together a join TFP Prime-Factor challenge team - work out some decent code, pool some resources and win some cash?! Any interest guys?

03-14-2005, 01:58 PM	#6 (permalink)
zen_tom Guest	Either directly to the TFP, or towards funding a TFP get-together for all those involved. The PR alone should be enough to generate contributions.

03-14-2005, 01:59 PM	#7 (permalink)
zen_tom Guest	However, it's a more than just a tricky job to actually factor the numbers, I wouldn't start booking any flights just yet...

03-14-2005, 05:29 PM	#8 (permalink)
wombatman Insane	What zen_tom said. I've had 5 computers running pretty continuously for at least 2 weeks doing numbers. And Zen, I had just thought about doing the smaller arrays. I tried doing a 2-dimensional array and quickly found out that I do not have that much memory. And yeah, this is for the RSA thing. I honestly did not mean to mislead if anyone didn't know where this was coming from. One potential problem I can see with trying to a joint venture on this would coordinating who is responsible for what range of numbers and so on. Different speed and different types of processors will of course run at different speeds. There is also a somewhat noticeable difference in the time to check the 90+ digit numbers (I cut the 193 in half and am focusing on the lower half, since there must be one factor at or below 97 digits) and the 11 and 12 digit numbers (starting from say 3). For the time being, I'll continue working on it solo I guess. It should be noted, however, that I was giving careful consideration to a donation of some of the money to TFP as I came across the Erathosthenes(sp?) Sieve method here. Now, due to the size of the numbers involved, I'm having to rethink the details of the method, but I think a thousand or two thousand dollars wouldn't be out of the question, and I imagine that would help a good deal with bandwidth/server costs. Thanks again for the comments, Zen.

03-14-2005, 09:21 PM	#10 (permalink)
wombatman Insane	Well, mostly because Java is the class I'm in. I have a basic understanding of C++, but don't have a compiler for it currently and also don't quite know how to go about working with really large numbers (read 193 digit+). If someone could direct me somewhere that I could learn about handling a number that large, I would certainly give it a shot, especially since I would like to speed this up a bit.

03-15-2005, 04:43 AM	#12 (permalink)
zen_tom Guest	You might have to manufacture your own class of number, HugeInteger or something - not sure how I'd go about this, but it might be stored as an array of chars. Floating point numbers like floats and doubles can be innaccurate when they either have to store very large or very small numbers in a precise manner - which wont help in this problem. You were right first time with the BigInteger. Allocating a large array got a near 600 bit record size is of course going to simply going to up a fair amount of memory. The alternative is to store the numbers in memory using some constructed class that you've designed to optimise the processes you are running. For example, their might be some useful heuristics that will help strip out various factors, or maybe converting the number into decimal, you could run some kind of mask/pattern matching on it (I'm not 100% sure what I'm talking about here actually so I'll shut up quickly before I embarrass myself) Anyway, good luck and I'll have a think about how else we could approach this.

03-15-2005, 06:02 AM	#13 (permalink)
vinaur Insane	So are you going to eventually go through a bunch of primes and try to divide the given number by those primes??? Wombatman: you said that you split up the number into two parts and are working with the lower part. Well, when you do find the number that is a factor of the first half, it does not necessarily mean that it will also be a factor of the whole thing. You would have to test the factor against the whole number to determine whether it really is a factor. I was thinking more of the other way around. You look at the higher part first and when you find a factor, you check it against the lower part. Last edited by vinaur; 03-15-2005 at 06:08 AM..

03-15-2005, 09:33 AM	#14 (permalink)
wombatman Insane	Sorry, I meant that I am only looking for factors of the target number (the big 193 digit one) in numbers that are 97 digits long or less. Every time I find a prime I am dividing it into the main number (the big one) and seeing if it divides evenly. When I find the prime that does, I'm having the program write the other factor (whatever the big number divided by the prime is) into a text file and stopping. This way I have both factors.

03-17-2005, 08:29 PM	#15 (permalink)
Rangsk Crazy Location: San Diego, CA	I'd like to apologize, I recently did some searching and it turns out that in a lot of situations nowadays, Java is actually FASTER than C++ because of Just in Time compiling and Hotspot, etc. Java is no longer interpreted, instead the bytecode is compiled just before running - pretty cool if you ask me __________________ "Don't believe everything you read on the internet. Except this. Well, including this, I suppose." -- Douglas Adams

03-30-2005, 10:50 AM	#16 (permalink)
wombatman Insane	One more question since I'm still running this program...does displaying the prime numbers I find slow down the process significantly? And if so, would writing them to a file make a difference? The only reason I'm printing them all out is so I can stop the program at any time to free up CPU power.

03-30-2005, 11:07 AM	#17 (permalink)
zen_tom Guest	i/o of any kind (network, file - or even to the screen) will slow down the process - I'd try to cut it down to a minimum where possible.

03-30-2005, 11:14 AM	#18 (permalink)
a-j Tilted	Console I/O can slow down a program, but I don't think it would make a huge difference if you are only outputting a number every so often, if the console is scrolling like crazy, then it is probably being inhibited. If the number is being output in hex/binary I would think it is resonably fast, if it is doing it in base 10, then it would probably be slower.

03-30-2005, 12:37 PM	#19 (permalink)
wombatman Insane	Ok, well I'm printing out every prime the program finds (not prime factors since they're what I'm actually trying to discover), so I guess I should try not outputting them to the screen and maybe writing every 500th prime or so to a file so if something does happen, I'm not too far behind. And if it matters, I'm printing out a string to the console since that's how the BigInteger class seems to handle the long numbers.