Greg Smith mentioned to me yesterday that he's seen that kink between 16 and 32 cores on other tests that have nothing to do with PostgreSQL, such as STREAM, and that he believes it has to do with Linux's scheduler behavior.
For the curious, this machine is running Linux 3.2. Previous benchmarking efforts on a different 64-core server running an earlier version of PostgreSQL 9.2devel and an older Linux kernel were not a success, but the lseek scalability changes in 3.2 seem to have made a big difference, and the newer PostgreSQL code has some changes that mitigate ProcArrayLock contention.
Good news. Great work Robert
ReplyDeletegreat stuff :)
ReplyDeleteIt sounds like you attribute the big performance jump in 9.2 vs. 9.1 mostly to Linux's changes. Is that correct, or did I misunderstand?
ReplyDeleteJoshua: No, I'm saying that to get good performance on a 64-core server, you're going to need PostgreSQL >= 9.2 *and* Linux >= 3.2. Most of the changes are actually on the PostgreSQL side, but the lseek scaling stuff in the Linux kernel was important, too.
ReplyDeleteRobert, can you comment on whether such improvements will be seen on Solaris 10 or not ? Any guesses , ideas ?
DeleteI use postgres 8.3 and it probably scales uptil 16-cores on solaris 10. If postgres 9.3 can push it up to 32 cores, I might as well switch to that.
Thanks for this post.
Thanks for this great post.
DeleteCan you comment on whether PostgresSQL >= 9.2 would scale as it does on linux >= 3.2 on solaris 10 ? I use postgres 8.3.7 on solaris 10, which in my experience, scales uptil 16 cores. If PostgresQL >= 9.2 scales as well as you've shown on solaris 10, I might as well switch to that.
Thomas, I have not tested on Solaris 10. It should be better than 8.3, but I don't know by how much.
DeleteHmm, makes me wonder how FreeBSD scales with this.
ReplyDeleteI think FreeBSD is still working on resolving some issues in the virtual memory subsystem that prevent it from scaling well at higher core counts. However, I'm hopeful that we'll see some positive news on that front later this year.
Deletedo you have results for a 3.0 or a 2.6 kernel?
ReplyDeleteYeah, definately need to test with FreeBSD.
ReplyDeleteRobert,
ReplyDeletePermission to use your graphs in an upcoming 9.2 presentation?
Josh, you're welcome to use them, with attribution.
ReplyDeleteHello Robert,
ReplyDeleteYou are lucky to have access to a 64-Core server!
Looking at your graph, PG serves 30k requests/sec. at concurrency 6.
At this concurrency, on a mere 6-Core CPU, G-WAN (our Web application server with C, C++, D, Objective-C and Java scripts) is processing:
Type of Request 6 Clients 1,000 Clients
-------------------- ------------------ ------------------
100-byte static file ...... 250k requests/sec. 720k requests/sec.
dynamic 100-year Loan ..... 150k requests/sec. 160k requests/sec.
http://gwan.ch/imgs/localhost_gwan2.10.8.png
http://gwan.ch/imgs/localhost_gwan2.10.10_loan%28100%29k.png
http://gwan.ch/en_apachebench_httperf.html#weighttp
http://gwan.ch/source/ab.c
http://gwan.ch/source/loan.c
Scaling vertically is mandatory in a world of multi-Core CPUs - but performing vertically makes vertical scalability fly (much higher).
I would be very willing to make tests with you about how G-WAN and PG can bring this game a step higher (contact me at my first name @trustleap.com).
Sincerely,
Pierre.
Related benchmarks: http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved — but Linux was faster anyway
ReplyDeleteI'm not certain, but I believe each AMD 6272 processor has 16 cores. If so, the title of the chart probably should read "4 x 16-core AMD 6272 processors".
ReplyDeleteRobert
ReplyDeleteWhat about FreeBSS performance scalability? Is it linear as well?
Can you publish the raw numbers for TPS vs # Clients? Something looks strange between 20 and 36 clients for the PG 9.2 Current curve.
ReplyDeletenclients pg91 pg92devel
Delete1 5126.256565 5606.254711
2 9875.569885 10024.552448
4 18770.215456 20058.372291
8 38556.199226 42486.574505
12 53542.256900 62905.209873
16 64716.084974 83002.638514
20 71026.444659 98034.531842
24 74377.904306 115449.670165
28 74436.084120 133433.767200
32 72830.033132 173886.541815
36 70592.447148 228249.257321
40 68150.060935 255120.364728
44 65814.140755 277912.139404
48 63792.179169 297396.741853
52 62674.478417 312949.733691
56 60771.345728 327812.113433
60 59543.504487 340869.668340
64 57924.077125 354613.280879
68 56070.311028 357854.846415
72 55275.496822 358128.415224
80 52899.178807 357011.082207
Oracle migration to EDB using EDB studio tool seems fine, but all data types seems not supported, is there a plan to incoperate that too in future ? so that we can simply do Oracle migration easily
ReplyDeletePlease use http://forums.enterprisedb.com/forums/list.page for support on EnterpriseDB products and services, or consult your account manager for pricing and product plans. Unfortunately, I'm not able to comment on those issues here.
Deleteso cool!
ReplyDeleteI have a 64-core freebsd server and would be willing to run the test suite on it.
ReplyDeleteGo for it Reed, i'm sure many await your results!
ReplyDeleteHi Pierre Gauthier,
ReplyDeleteShow us the code!
Hi Robert
ReplyDeleteI know this post is a year old now - however I have at my disposal at 64 core IBM Power server running Linux. Would be more than happy (and interested) to see how performance compares on Power architecture - also happy to collaborate if there's anything I can use this server to help with.
Many thanks,
James