Optimizing for Modulo QueriesCan MySQL reasonably perform queries on billions of rows?MySQL Read Speed and...
Why is working on the same position for more than 15 years not a red flag?
How do you funnel food off a cutting board?
Do authors have to be politically correct in article-writing?
Explain the objections to these measures against human trafficking
If I delete my router's history can my ISP still provide it to my parents?
How experienced do I need to be to go on a photography workshop?
Could flying insects re-enter the Earth's atmosphere from space without burning up?
Eww, those bytes are gross
Planet at the end of Solo: A Star Wars Story
Why can a 352GB NumPy ndarray be used on an 8GB memory macOS computer?
Why do neural networks need so many training examples to perform?
The vanishing of sum of coefficients: symmetric polynomials
How to remove trailing forward slash
Number of FLOP (Floating Point Operations) for exponentiation
Avoiding morning and evening handshakes
Manipulating a general length function
Dilemma of explaining to interviewer that he is the reason for declining second interview
Is it safe to try charging my laptop with a USB-C PD charger that has less wattage than recommended?
How does Internet communication work on a coaxial cable?
Closed form for these polynomials?
It took me a lot of time to make this, pls like. (YouTube Comments #1)
Why don't I see the difference between two different files in insert mode in vim?
How to avoid being sexist when trying to employ someone to function in a very sexist environment?
Why zero tolerance on nudity in space?
Optimizing for Modulo Queries
Can MySQL reasonably perform queries on billions of rows?MySQL Read Speed and Partitioning on Separate DrivesOptimizing multiple MySQL sub-queries and joins for multiple runsMySQL join query indexesHigh volume data aggregationOptimizing SELECT queriesoptimizing query for slow queriesChoosing index fillfactor for large, highly inserted tableRegularly updating table with huge amount of rowsWhich data model/schema to apply to a timeseries datastore for data sources with varying fields
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
bumped to the homepage by Community♦ 2 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
bumped to the homepage by Community♦ 2 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
1011
bumped to the homepage by Community♦ 2 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 2 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f84841%2foptimizing-for-modulo-queries%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
edited Dec 10 '14 at 20:40
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f84841%2foptimizing-for-modulo-queries%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
