Pairwise combinations of filenames
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
add a comment |
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
Tryxargs -n
.
– ctrl-alt-delor
Dec 23 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 at 19:38
add a comment |
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
bash
edited Dec 23 at 22:30
asked Dec 23 at 18:59
rmf
292212
292212
Tryxargs -n
.
– ctrl-alt-delor
Dec 23 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 at 19:38
add a comment |
Tryxargs -n
.
– ctrl-alt-delor
Dec 23 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 at 19:38
Try
xargs -n
.– ctrl-alt-delor
Dec 23 at 19:14
Try
xargs -n
.– ctrl-alt-delor
Dec 23 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 at 19:38
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 at 19:38
add a comment |
4 Answers
4
active
oldest
votes
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
1
Note that it is better to useprintf
rather thanecho
: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn
, instead ofecho
orprintf
.echo
works fine as an example here, though.
– ilkkachu
Dec 23 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
Yes, but without knowing the cost offn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 at 20:16
add a comment |
With join
trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-j
option points to a common field position to join on; but-j9999
will provoke mixed joining resembling cartesian product.
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490649%2fpairwise-combinations-of-filenames%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
1
Note that it is better to useprintf
rather thanecho
: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn
, instead ofecho
orprintf
.echo
works fine as an example here, though.
– ilkkachu
Dec 23 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
add a comment |
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
1
Note that it is better to useprintf
rather thanecho
: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn
, instead ofecho
orprintf
.echo
works fine as an example here, though.
– ilkkachu
Dec 23 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
add a comment |
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
answered Dec 23 at 19:56
ilkkachu
55.5k783151
55.5k783151
1
Note that it is better to useprintf
rather thanecho
: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn
, instead ofecho
orprintf
.echo
works fine as an example here, though.
– ilkkachu
Dec 23 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
add a comment |
1
Note that it is better to useprintf
rather thanecho
: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn
, instead ofecho
orprintf
.echo
works fine as an example here, though.
– ilkkachu
Dec 23 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
1
1
Note that it is better to use
printf
rather than echo
: unix.stackexchange.com/questions/65803/…– cryptarch
Dec 23 at 20:06
Note that it is better to use
printf
rather than echo
: unix.stackexchange.com/questions/65803/…– cryptarch
Dec 23 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call to
fn
, instead of echo
or printf
. echo
works fine as an example here, though.– ilkkachu
Dec 23 at 21:24
@cryptarch, to be in line with the question, the content of the loop should be a call to
fn
, instead of echo
or printf
. echo
works fine as an example here, though.– ilkkachu
Dec 23 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 at 21:45
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
Yes, but without knowing the cost offn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 at 20:16
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
Yes, but without knowing the cost offn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 at 20:16
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
answered Dec 23 at 20:05
Stephen Harris
24.6k24477
24.6k24477
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
Yes, but without knowing the cost offn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 at 20:16
add a comment |
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
Yes, but without knowing the cost offn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 at 20:16
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 at 20:08
Yes, but without knowing the cost of
fn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call to fn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.– Stephen Harris
Dec 23 at 20:16
Yes, but without knowing the cost of
fn
it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call to fn
takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.– Stephen Harris
Dec 23 at 20:16
add a comment |
With join
trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-j
option points to a common field position to join on; but-j9999
will provoke mixed joining resembling cartesian product.
add a comment |
With join
trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-j
option points to a common field position to join on; but-j9999
will provoke mixed joining resembling cartesian product.
add a comment |
With join
trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-j
option points to a common field position to join on; but-j9999
will provoke mixed joining resembling cartesian product.
With join
trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-j
option points to a common field position to join on; but-j9999
will provoke mixed joining resembling cartesian product.
answered Dec 23 at 19:36
RomanPerekhrest
22.8k12346
22.8k12346
add a comment |
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
answered Dec 25 at 21:43
Ole Tange
12k1451105
12k1451105
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490649%2fpairwise-combinations-of-filenames%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Try
xargs -n
.– ctrl-alt-delor
Dec 23 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 at 19:38