Comments on: Large Binary Data is (not) a Weakness of Erlang http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/ Computer Languages, Programming, and Free Software Mon, 06 Apr 2009 02:59:08 +0000 http://wordpress.com/ hourly 1 By: Philip Robinson http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-30 Philip Robinson Thu, 12 Jul 2007 12:17:48 +0000 http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-30 > the original blog author should be happy. I am indeed. :-) I posted that entry on the off chance that someone else might run into this rare problem and need a quick solution. Having the issue treated as a bug and fixed in the language itself was a most pleasant surprise. > the original blog author should be happy.

I am indeed. :-)

I posted that entry on the off chance that someone else might run into this rare problem and need a quick solution. Having the issue treated as a bug and fixed in the language itself was a most pleasant surprise.

]]> By: Top Posts « WordPress.com http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-29 Top Posts « WordPress.com Wed, 11 Jul 2007 00:02:46 +0000 http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-29 [...] Large Binary Data is (not) a Weakness of Erlang Update: Good news in the comments, with some good details. To sum up: split_binary/2 is the call to use. Binaries are […] [...] [...] Large Binary Data is (not) a Weakness of Erlang Update: Good news in the comments, with some good details. To sum up: split_binary/2 is the call to use. Binaries are […] [...]

]]> By: Per Gustafsson http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-28 Per Gustafsson Tue, 10 Jul 2007 14:41:03 +0000 http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-28 Selecting slices in binaries is O(1). The reason there were problems to skip ahead in binaries was because binary pattern matching did not handle bignums correctly. This will be fixed in the next release. There is a danger in referencing binaries which can lead to wasted memory as you mentioned. As far as I know the gc does nothing to fix this. There is already built in functions to handle binaries e.g. split_binary/2. Which divides a binary into two at O(1) cost. If you want to have large binaries handled correctly now you should use a 64-bit machine, this holds true even if pattern matching is fixed. Because the sizes of binaries on 32-bit machines is limited. Definitely to smaller than 4 GB, but the limit is perhaps even smaller than that. Selecting slices in binaries is O(1). The reason there were problems to skip ahead in binaries was because binary pattern matching did not handle bignums correctly. This will be fixed in the next release.

There is a danger in referencing binaries which can lead to wasted memory as you mentioned. As far as I know the gc does nothing to fix this.

There is already built in functions to handle binaries e.g. split_binary/2. Which divides a binary into two at O(1) cost.

If you want to have large binaries handled correctly now you should use a 64-bit machine, this holds true even if pattern matching is fixed. Because the sizes of binaries on 32-bit machines is limited. Definitely to smaller than 4 GB, but the limit is perhaps even smaller than that.

]]> By: Per Melin http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-27 Per Melin Tue, 10 Jul 2007 12:18:00 +0000 http://metalinguist.wordpress.com/2007/07/09/largbe-binary-data-is-a-weakness-of-erlang/#comment-27 You can create a binary as a pointer into another binary: {_, B2} = split_binary(B1, 4711). And, from the Erlang FAQ: "Normal data in Erlang is put on the process heap, which is garbage collected. Large binaries, on the other hand, are reference counted." You can create a binary as a pointer into another binary:

{_, B2} = split_binary(B1, 4711).

And, from the Erlang FAQ:
“Normal data in Erlang is put on the process heap, which is garbage collected. Large binaries, on the other hand, are reference counted.”

]]>