What does Turing mean by this statement?












16












$begingroup$


I encountered below statement by Alan M. Turing at here:




"The view that machines cannot give rise to surprises is due, I
believe, to a fallacy to which philosophers and mathematicians are
particularly subject. This is the assumption that as soon as a fact is
presented to a mind all consequences of that fact spring into the mind
simultaneously with it. It is a very useful assumption under many
circumstances, but one too easily forgets that it is false."




I am not a native English speaker. Could anyone explain it in plain English? Thanks!










share|cite|improve this question







New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$








  • 1




    $begingroup$
    perhaps, it's better suited for philosophy portal rather to hard science like CS
    $endgroup$
    – Bulat
    18 hours ago






  • 2




    $begingroup$
    @Bulat I was going to say the same -- and redirect to English Language Learners -- but then I realised that there is some CS-related content that can be explained in an answer, which probably wouldn't be picked up on, in other parts of Stack Exchange.
    $endgroup$
    – David Richerby
    18 hours ago






  • 3




    $begingroup$
    A good example is iteration of the transformation z := z² + c, where z and c are complex numbers. What happens if I take any starting point on the plane z and iterate, will the number go to infinity or not? An ordinary fellow would say, yeah, this will give you two regions or maybe a few more where the value goes to zero and for the rest it goes to infinity. Relatively unsurprising. Then Mandelbrot comes along and actually plots the regions on the the plane defined by this simple "machine". As the result comes out of the dotmatrix printer, this simple "machine" proves itself ... weird.
    $endgroup$
    – David Tonhofer
    11 hours ago












  • $begingroup$
    Facebook and other social media are a great example of this... A lot of the consequences of their algorithms are not something that was expected by the creators (or anyone really).
    $endgroup$
    – aslum
    7 hours ago










  • $begingroup$
    A rather quirky individual once referred to this using a fire metaphor: "The bigger you build your bonfire of knowledge, the more darkness is revealed to your startled eye"
    $endgroup$
    – JacobIRR
    6 hours ago
















16












$begingroup$


I encountered below statement by Alan M. Turing at here:




"The view that machines cannot give rise to surprises is due, I
believe, to a fallacy to which philosophers and mathematicians are
particularly subject. This is the assumption that as soon as a fact is
presented to a mind all consequences of that fact spring into the mind
simultaneously with it. It is a very useful assumption under many
circumstances, but one too easily forgets that it is false."




I am not a native English speaker. Could anyone explain it in plain English? Thanks!










share|cite|improve this question







New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$








  • 1




    $begingroup$
    perhaps, it's better suited for philosophy portal rather to hard science like CS
    $endgroup$
    – Bulat
    18 hours ago






  • 2




    $begingroup$
    @Bulat I was going to say the same -- and redirect to English Language Learners -- but then I realised that there is some CS-related content that can be explained in an answer, which probably wouldn't be picked up on, in other parts of Stack Exchange.
    $endgroup$
    – David Richerby
    18 hours ago






  • 3




    $begingroup$
    A good example is iteration of the transformation z := z² + c, where z and c are complex numbers. What happens if I take any starting point on the plane z and iterate, will the number go to infinity or not? An ordinary fellow would say, yeah, this will give you two regions or maybe a few more where the value goes to zero and for the rest it goes to infinity. Relatively unsurprising. Then Mandelbrot comes along and actually plots the regions on the the plane defined by this simple "machine". As the result comes out of the dotmatrix printer, this simple "machine" proves itself ... weird.
    $endgroup$
    – David Tonhofer
    11 hours ago












  • $begingroup$
    Facebook and other social media are a great example of this... A lot of the consequences of their algorithms are not something that was expected by the creators (or anyone really).
    $endgroup$
    – aslum
    7 hours ago










  • $begingroup$
    A rather quirky individual once referred to this using a fire metaphor: "The bigger you build your bonfire of knowledge, the more darkness is revealed to your startled eye"
    $endgroup$
    – JacobIRR
    6 hours ago














16












16








16





$begingroup$


I encountered below statement by Alan M. Turing at here:




"The view that machines cannot give rise to surprises is due, I
believe, to a fallacy to which philosophers and mathematicians are
particularly subject. This is the assumption that as soon as a fact is
presented to a mind all consequences of that fact spring into the mind
simultaneously with it. It is a very useful assumption under many
circumstances, but one too easily forgets that it is false."




I am not a native English speaker. Could anyone explain it in plain English? Thanks!










share|cite|improve this question







New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$




I encountered below statement by Alan M. Turing at here:




"The view that machines cannot give rise to surprises is due, I
believe, to a fallacy to which philosophers and mathematicians are
particularly subject. This is the assumption that as soon as a fact is
presented to a mind all consequences of that fact spring into the mind
simultaneously with it. It is a very useful assumption under many
circumstances, but one too easily forgets that it is false."




I am not a native English speaker. Could anyone explain it in plain English? Thanks!







turing-machines computability computation-models






share|cite|improve this question







New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|cite|improve this question







New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|cite|improve this question




share|cite|improve this question






New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 20 hours ago









smwikipediasmwikipedia

1815




1815




New contributor




smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






smwikipedia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








  • 1




    $begingroup$
    perhaps, it's better suited for philosophy portal rather to hard science like CS
    $endgroup$
    – Bulat
    18 hours ago






  • 2




    $begingroup$
    @Bulat I was going to say the same -- and redirect to English Language Learners -- but then I realised that there is some CS-related content that can be explained in an answer, which probably wouldn't be picked up on, in other parts of Stack Exchange.
    $endgroup$
    – David Richerby
    18 hours ago






  • 3




    $begingroup$
    A good example is iteration of the transformation z := z² + c, where z and c are complex numbers. What happens if I take any starting point on the plane z and iterate, will the number go to infinity or not? An ordinary fellow would say, yeah, this will give you two regions or maybe a few more where the value goes to zero and for the rest it goes to infinity. Relatively unsurprising. Then Mandelbrot comes along and actually plots the regions on the the plane defined by this simple "machine". As the result comes out of the dotmatrix printer, this simple "machine" proves itself ... weird.
    $endgroup$
    – David Tonhofer
    11 hours ago












  • $begingroup$
    Facebook and other social media are a great example of this... A lot of the consequences of their algorithms are not something that was expected by the creators (or anyone really).
    $endgroup$
    – aslum
    7 hours ago










  • $begingroup$
    A rather quirky individual once referred to this using a fire metaphor: "The bigger you build your bonfire of knowledge, the more darkness is revealed to your startled eye"
    $endgroup$
    – JacobIRR
    6 hours ago














  • 1




    $begingroup$
    perhaps, it's better suited for philosophy portal rather to hard science like CS
    $endgroup$
    – Bulat
    18 hours ago






  • 2




    $begingroup$
    @Bulat I was going to say the same -- and redirect to English Language Learners -- but then I realised that there is some CS-related content that can be explained in an answer, which probably wouldn't be picked up on, in other parts of Stack Exchange.
    $endgroup$
    – David Richerby
    18 hours ago






  • 3




    $begingroup$
    A good example is iteration of the transformation z := z² + c, where z and c are complex numbers. What happens if I take any starting point on the plane z and iterate, will the number go to infinity or not? An ordinary fellow would say, yeah, this will give you two regions or maybe a few more where the value goes to zero and for the rest it goes to infinity. Relatively unsurprising. Then Mandelbrot comes along and actually plots the regions on the the plane defined by this simple "machine". As the result comes out of the dotmatrix printer, this simple "machine" proves itself ... weird.
    $endgroup$
    – David Tonhofer
    11 hours ago












  • $begingroup$
    Facebook and other social media are a great example of this... A lot of the consequences of their algorithms are not something that was expected by the creators (or anyone really).
    $endgroup$
    – aslum
    7 hours ago










  • $begingroup$
    A rather quirky individual once referred to this using a fire metaphor: "The bigger you build your bonfire of knowledge, the more darkness is revealed to your startled eye"
    $endgroup$
    – JacobIRR
    6 hours ago








1




1




$begingroup$
perhaps, it's better suited for philosophy portal rather to hard science like CS
$endgroup$
– Bulat
18 hours ago




$begingroup$
perhaps, it's better suited for philosophy portal rather to hard science like CS
$endgroup$
– Bulat
18 hours ago




2




2




$begingroup$
@Bulat I was going to say the same -- and redirect to English Language Learners -- but then I realised that there is some CS-related content that can be explained in an answer, which probably wouldn't be picked up on, in other parts of Stack Exchange.
$endgroup$
– David Richerby
18 hours ago




$begingroup$
@Bulat I was going to say the same -- and redirect to English Language Learners -- but then I realised that there is some CS-related content that can be explained in an answer, which probably wouldn't be picked up on, in other parts of Stack Exchange.
$endgroup$
– David Richerby
18 hours ago




3




3




$begingroup$
A good example is iteration of the transformation z := z² + c, where z and c are complex numbers. What happens if I take any starting point on the plane z and iterate, will the number go to infinity or not? An ordinary fellow would say, yeah, this will give you two regions or maybe a few more where the value goes to zero and for the rest it goes to infinity. Relatively unsurprising. Then Mandelbrot comes along and actually plots the regions on the the plane defined by this simple "machine". As the result comes out of the dotmatrix printer, this simple "machine" proves itself ... weird.
$endgroup$
– David Tonhofer
11 hours ago






$begingroup$
A good example is iteration of the transformation z := z² + c, where z and c are complex numbers. What happens if I take any starting point on the plane z and iterate, will the number go to infinity or not? An ordinary fellow would say, yeah, this will give you two regions or maybe a few more where the value goes to zero and for the rest it goes to infinity. Relatively unsurprising. Then Mandelbrot comes along and actually plots the regions on the the plane defined by this simple "machine". As the result comes out of the dotmatrix printer, this simple "machine" proves itself ... weird.
$endgroup$
– David Tonhofer
11 hours ago














$begingroup$
Facebook and other social media are a great example of this... A lot of the consequences of their algorithms are not something that was expected by the creators (or anyone really).
$endgroup$
– aslum
7 hours ago




$begingroup$
Facebook and other social media are a great example of this... A lot of the consequences of their algorithms are not something that was expected by the creators (or anyone really).
$endgroup$
– aslum
7 hours ago












$begingroup$
A rather quirky individual once referred to this using a fire metaphor: "The bigger you build your bonfire of knowledge, the more darkness is revealed to your startled eye"
$endgroup$
– JacobIRR
6 hours ago




$begingroup$
A rather quirky individual once referred to this using a fire metaphor: "The bigger you build your bonfire of knowledge, the more darkness is revealed to your startled eye"
$endgroup$
– JacobIRR
6 hours ago










5 Answers
5






active

oldest

votes


















18












$begingroup$


Mathematicians and philosophers often assume that machines (and here, he probably means "computers") cannot surprise us. This is because they assume that once we learn some fact, we immediately understand every consequence of this fact. This is often a useful assumption, but it's easy to forget that it's false.




He's saying that systems with simple, finite descriptions (e.g., Turing machines) can exhibit very complicated behaviour and that this surprises some people. We can easily understand the concept of Turing machines but then we realise that they have complicated consequences, such as the undecidability of the halting problem and so on. The technical term here is that "knowledge is not closed under deduction". That is, we can know some fact $A$, but not know $B$, even though $A$ implies $B$.



Honestly, though, I'm not sure that Turing's argument is very good. Perhaps I have the benefit of writing nearly 70 years after Turing, and my understanding is that the typical mathematician knows much more about mathematical logic than they did in Turing's time. But it seems to me that mathematicians are mostly quite familiar with the idea of simple systems having complex behaviour. For example, every mathematician knows the definition of a group, which consists of just four simple axioms. But nobody – today or then – would think, "Aha. I know the four axioms, therefore I know every fact about groups." Similarly, Peano's axioms give a very short description of the natural numbers but nobody who reads them thinks "Right, I know every theorem about the natural numbers, now. Let's move on to something else."






share|cite|improve this answer











$endgroup$









  • 10




    $begingroup$
    Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
    $endgroup$
    – BurnsBA
    15 hours ago






  • 4




    $begingroup$
    I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
    $endgroup$
    – alephzero
    9 hours ago








  • 1




    $begingroup$
    @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
    $endgroup$
    – David Richerby
    9 hours ago






  • 2




    $begingroup$
    Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
    $endgroup$
    – Derek Elkins
    4 hours ago












  • $begingroup$
    It is the absence of epistemic closure.
    $endgroup$
    – Dan D.
    3 hours ago



















13












$begingroup$

Just an example - given chess rules, anyone should immediately figure the best strategy to play chess.



Of course, it doesn't work. Even people aren't equal, and computers may outperform us due to their better abilities to make conclusions from the facts.






share|cite|improve this answer









$endgroup$





















    4












    $begingroup$

    People might assume that if I write a program, and I understand the algorithm completely, and there are no bugs, then I should know what the output of that program would be, and that it should not surprise me.



    Turing says (and I agree) that this is not the case: The output can be surprising. The solution to a travelling salesman problem can be surprising. The best way to build a full adder can be surprising. The best move in a chess game can be surprising.






    share|cite|improve this answer









    $endgroup$













    • $begingroup$
      This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
      $endgroup$
      – Discrete lizard
      13 hours ago



















    3












    $begingroup$

    This is the idea of emergence, which is when complex behavior of a result results from the interaction of relatively simple rules. There are lots of examples of this in nature, as that link points out. Insect colonies, bird flocks, schools of fish, and of course, consciousness. In a flock of birds or school of fish, each individual in the swam is only making decisions based on the others immediately surrounding them, but when you put a bunch of those individuals together all following those rules, you start to see more coordinated behavior than you'd expect without a higher level plan. If you go on Youtube and watch demonstrations of robot swarms, you see that they all avoid hitting each other and work in unison. Surprisingly this doesn't need to be accomplished by having a single central computer coordinate the behavior of each individual robot but can instead be done using swarm robotics where, like the insects or the birds or the fish, each robot is making local decisions which leads to emergent coordination.



    Another interesting demonstration of emergent behavior is Conway's Game of Life. The rules for the game are extremely simple, but can lead to very fascinating results



    A tempting argument against the ability of computers to gain human-intelligence is to say that since they can only do precisely what they're programmed to do, they must only exhibit the intelligence that we program them with. If this were true, then we would also not expect the relatively simple behavior of neurons to give rise to human intelligence. Yet as far as we can tell, this IS the case and consciousness is an emergent property of neural processing. I'm sure Turing would have loved to see what's become possible today with the use of artificial neural networks






    share|cite|improve this answer










    New contributor




    mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.






    $endgroup$





















      -2












      $begingroup$

      Surprise is something you did not expect. Turing says once you see what the machine says, you may be surprised.






      share|cite|improve this answer








      New contributor




      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      $endgroup$









      • 3




        $begingroup$
        This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
        $endgroup$
        – Discrete lizard
        13 hours ago












      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "419"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });






      smwikipedia is a new contributor. Be nice, and check out our Code of Conduct.










      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcs.stackexchange.com%2fquestions%2f107198%2fwhat-does-turing-mean-by-this-statement%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      5 Answers
      5






      active

      oldest

      votes








      5 Answers
      5






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      18












      $begingroup$


      Mathematicians and philosophers often assume that machines (and here, he probably means "computers") cannot surprise us. This is because they assume that once we learn some fact, we immediately understand every consequence of this fact. This is often a useful assumption, but it's easy to forget that it's false.




      He's saying that systems with simple, finite descriptions (e.g., Turing machines) can exhibit very complicated behaviour and that this surprises some people. We can easily understand the concept of Turing machines but then we realise that they have complicated consequences, such as the undecidability of the halting problem and so on. The technical term here is that "knowledge is not closed under deduction". That is, we can know some fact $A$, but not know $B$, even though $A$ implies $B$.



      Honestly, though, I'm not sure that Turing's argument is very good. Perhaps I have the benefit of writing nearly 70 years after Turing, and my understanding is that the typical mathematician knows much more about mathematical logic than they did in Turing's time. But it seems to me that mathematicians are mostly quite familiar with the idea of simple systems having complex behaviour. For example, every mathematician knows the definition of a group, which consists of just four simple axioms. But nobody – today or then – would think, "Aha. I know the four axioms, therefore I know every fact about groups." Similarly, Peano's axioms give a very short description of the natural numbers but nobody who reads them thinks "Right, I know every theorem about the natural numbers, now. Let's move on to something else."






      share|cite|improve this answer











      $endgroup$









      • 10




        $begingroup$
        Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
        $endgroup$
        – BurnsBA
        15 hours ago






      • 4




        $begingroup$
        I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
        $endgroup$
        – alephzero
        9 hours ago








      • 1




        $begingroup$
        @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
        $endgroup$
        – David Richerby
        9 hours ago






      • 2




        $begingroup$
        Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
        $endgroup$
        – Derek Elkins
        4 hours ago












      • $begingroup$
        It is the absence of epistemic closure.
        $endgroup$
        – Dan D.
        3 hours ago
















      18












      $begingroup$


      Mathematicians and philosophers often assume that machines (and here, he probably means "computers") cannot surprise us. This is because they assume that once we learn some fact, we immediately understand every consequence of this fact. This is often a useful assumption, but it's easy to forget that it's false.




      He's saying that systems with simple, finite descriptions (e.g., Turing machines) can exhibit very complicated behaviour and that this surprises some people. We can easily understand the concept of Turing machines but then we realise that they have complicated consequences, such as the undecidability of the halting problem and so on. The technical term here is that "knowledge is not closed under deduction". That is, we can know some fact $A$, but not know $B$, even though $A$ implies $B$.



      Honestly, though, I'm not sure that Turing's argument is very good. Perhaps I have the benefit of writing nearly 70 years after Turing, and my understanding is that the typical mathematician knows much more about mathematical logic than they did in Turing's time. But it seems to me that mathematicians are mostly quite familiar with the idea of simple systems having complex behaviour. For example, every mathematician knows the definition of a group, which consists of just four simple axioms. But nobody – today or then – would think, "Aha. I know the four axioms, therefore I know every fact about groups." Similarly, Peano's axioms give a very short description of the natural numbers but nobody who reads them thinks "Right, I know every theorem about the natural numbers, now. Let's move on to something else."






      share|cite|improve this answer











      $endgroup$









      • 10




        $begingroup$
        Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
        $endgroup$
        – BurnsBA
        15 hours ago






      • 4




        $begingroup$
        I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
        $endgroup$
        – alephzero
        9 hours ago








      • 1




        $begingroup$
        @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
        $endgroup$
        – David Richerby
        9 hours ago






      • 2




        $begingroup$
        Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
        $endgroup$
        – Derek Elkins
        4 hours ago












      • $begingroup$
        It is the absence of epistemic closure.
        $endgroup$
        – Dan D.
        3 hours ago














      18












      18








      18





      $begingroup$


      Mathematicians and philosophers often assume that machines (and here, he probably means "computers") cannot surprise us. This is because they assume that once we learn some fact, we immediately understand every consequence of this fact. This is often a useful assumption, but it's easy to forget that it's false.




      He's saying that systems with simple, finite descriptions (e.g., Turing machines) can exhibit very complicated behaviour and that this surprises some people. We can easily understand the concept of Turing machines but then we realise that they have complicated consequences, such as the undecidability of the halting problem and so on. The technical term here is that "knowledge is not closed under deduction". That is, we can know some fact $A$, but not know $B$, even though $A$ implies $B$.



      Honestly, though, I'm not sure that Turing's argument is very good. Perhaps I have the benefit of writing nearly 70 years after Turing, and my understanding is that the typical mathematician knows much more about mathematical logic than they did in Turing's time. But it seems to me that mathematicians are mostly quite familiar with the idea of simple systems having complex behaviour. For example, every mathematician knows the definition of a group, which consists of just four simple axioms. But nobody – today or then – would think, "Aha. I know the four axioms, therefore I know every fact about groups." Similarly, Peano's axioms give a very short description of the natural numbers but nobody who reads them thinks "Right, I know every theorem about the natural numbers, now. Let's move on to something else."






      share|cite|improve this answer











      $endgroup$




      Mathematicians and philosophers often assume that machines (and here, he probably means "computers") cannot surprise us. This is because they assume that once we learn some fact, we immediately understand every consequence of this fact. This is often a useful assumption, but it's easy to forget that it's false.




      He's saying that systems with simple, finite descriptions (e.g., Turing machines) can exhibit very complicated behaviour and that this surprises some people. We can easily understand the concept of Turing machines but then we realise that they have complicated consequences, such as the undecidability of the halting problem and so on. The technical term here is that "knowledge is not closed under deduction". That is, we can know some fact $A$, but not know $B$, even though $A$ implies $B$.



      Honestly, though, I'm not sure that Turing's argument is very good. Perhaps I have the benefit of writing nearly 70 years after Turing, and my understanding is that the typical mathematician knows much more about mathematical logic than they did in Turing's time. But it seems to me that mathematicians are mostly quite familiar with the idea of simple systems having complex behaviour. For example, every mathematician knows the definition of a group, which consists of just four simple axioms. But nobody – today or then – would think, "Aha. I know the four axioms, therefore I know every fact about groups." Similarly, Peano's axioms give a very short description of the natural numbers but nobody who reads them thinks "Right, I know every theorem about the natural numbers, now. Let's move on to something else."







      share|cite|improve this answer














      share|cite|improve this answer



      share|cite|improve this answer








      edited 15 hours ago

























      answered 18 hours ago









      David RicherbyDavid Richerby

      70.6k16107198




      70.6k16107198








      • 10




        $begingroup$
        Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
        $endgroup$
        – BurnsBA
        15 hours ago






      • 4




        $begingroup$
        I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
        $endgroup$
        – alephzero
        9 hours ago








      • 1




        $begingroup$
        @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
        $endgroup$
        – David Richerby
        9 hours ago






      • 2




        $begingroup$
        Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
        $endgroup$
        – Derek Elkins
        4 hours ago












      • $begingroup$
        It is the absence of epistemic closure.
        $endgroup$
        – Dan D.
        3 hours ago














      • 10




        $begingroup$
        Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
        $endgroup$
        – BurnsBA
        15 hours ago






      • 4




        $begingroup$
        I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
        $endgroup$
        – alephzero
        9 hours ago








      • 1




        $begingroup$
        @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
        $endgroup$
        – David Richerby
        9 hours ago






      • 2




        $begingroup$
        Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
        $endgroup$
        – Derek Elkins
        4 hours ago












      • $begingroup$
        It is the absence of epistemic closure.
        $endgroup$
        – Dan D.
        3 hours ago








      10




      10




      $begingroup$
      Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
      $endgroup$
      – BurnsBA
      15 hours ago




      $begingroup$
      Historically, the early 20th century had a strong academic belief in "solving" mathematics. E.g., Hilbert's program, and Whitehead+Russel's Principia Mathematica. Godel's work resolved that quest negatively, but I imagine it took some time for academia to fully embrace this notion; even fully acknowledging the correctness of Godel, people would still remember the grand ideas of Hilbert. I think Turing writing only two decades after Godel would be addressing his audience with this context in mind.
      $endgroup$
      – BurnsBA
      15 hours ago




      4




      4




      $begingroup$
      I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
      $endgroup$
      – alephzero
      9 hours ago






      $begingroup$
      I would question whether most mathematicians know "much more about mathematical logic" than Turing did. But it is obvious that almost all contemporary humans have vastly more practical experience of what machines (and particularly computers) can do than he did.
      $endgroup$
      – alephzero
      9 hours ago






      1




      1




      $begingroup$
      @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
      $endgroup$
      – David Richerby
      9 hours ago




      $begingroup$
      @alephzero That's not what I said! I said that the average mathematician today knows more about mathematical logic than the average mathematician during Turing's time.
      $endgroup$
      – David Richerby
      9 hours ago




      2




      2




      $begingroup$
      Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
      $endgroup$
      – Derek Elkins
      4 hours ago






      $begingroup$
      Your argument seems to be not that Turing's argument isn't good, but that it is unnecessary or directed at a strawman. I strongly suspect Turing had real people make arguments like this to him, so I don't think he's making a strawman out of nothing. As Discrete lizard states in a comment, Turing is only saying that a particular argument against machines surprising us is bad. Your answer just says that that this argument is bad has become even more obvious over time. That said, people (though usually not experts) still make arguments in this vein today.
      $endgroup$
      – Derek Elkins
      4 hours ago














      $begingroup$
      It is the absence of epistemic closure.
      $endgroup$
      – Dan D.
      3 hours ago




      $begingroup$
      It is the absence of epistemic closure.
      $endgroup$
      – Dan D.
      3 hours ago











      13












      $begingroup$

      Just an example - given chess rules, anyone should immediately figure the best strategy to play chess.



      Of course, it doesn't work. Even people aren't equal, and computers may outperform us due to their better abilities to make conclusions from the facts.






      share|cite|improve this answer









      $endgroup$


















        13












        $begingroup$

        Just an example - given chess rules, anyone should immediately figure the best strategy to play chess.



        Of course, it doesn't work. Even people aren't equal, and computers may outperform us due to their better abilities to make conclusions from the facts.






        share|cite|improve this answer









        $endgroup$
















          13












          13








          13





          $begingroup$

          Just an example - given chess rules, anyone should immediately figure the best strategy to play chess.



          Of course, it doesn't work. Even people aren't equal, and computers may outperform us due to their better abilities to make conclusions from the facts.






          share|cite|improve this answer









          $endgroup$



          Just an example - given chess rules, anyone should immediately figure the best strategy to play chess.



          Of course, it doesn't work. Even people aren't equal, and computers may outperform us due to their better abilities to make conclusions from the facts.







          share|cite|improve this answer












          share|cite|improve this answer



          share|cite|improve this answer










          answered 18 hours ago









          BulatBulat

          961512




          961512























              4












              $begingroup$

              People might assume that if I write a program, and I understand the algorithm completely, and there are no bugs, then I should know what the output of that program would be, and that it should not surprise me.



              Turing says (and I agree) that this is not the case: The output can be surprising. The solution to a travelling salesman problem can be surprising. The best way to build a full adder can be surprising. The best move in a chess game can be surprising.






              share|cite|improve this answer









              $endgroup$













              • $begingroup$
                This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
                $endgroup$
                – Discrete lizard
                13 hours ago
















              4












              $begingroup$

              People might assume that if I write a program, and I understand the algorithm completely, and there are no bugs, then I should know what the output of that program would be, and that it should not surprise me.



              Turing says (and I agree) that this is not the case: The output can be surprising. The solution to a travelling salesman problem can be surprising. The best way to build a full adder can be surprising. The best move in a chess game can be surprising.






              share|cite|improve this answer









              $endgroup$













              • $begingroup$
                This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
                $endgroup$
                – Discrete lizard
                13 hours ago














              4












              4








              4





              $begingroup$

              People might assume that if I write a program, and I understand the algorithm completely, and there are no bugs, then I should know what the output of that program would be, and that it should not surprise me.



              Turing says (and I agree) that this is not the case: The output can be surprising. The solution to a travelling salesman problem can be surprising. The best way to build a full adder can be surprising. The best move in a chess game can be surprising.






              share|cite|improve this answer









              $endgroup$



              People might assume that if I write a program, and I understand the algorithm completely, and there are no bugs, then I should know what the output of that program would be, and that it should not surprise me.



              Turing says (and I agree) that this is not the case: The output can be surprising. The solution to a travelling salesman problem can be surprising. The best way to build a full adder can be surprising. The best move in a chess game can be surprising.







              share|cite|improve this answer












              share|cite|improve this answer



              share|cite|improve this answer










              answered 13 hours ago









              gnasher729gnasher729

              12.1k1318




              12.1k1318












              • $begingroup$
                This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
                $endgroup$
                – Discrete lizard
                13 hours ago


















              • $begingroup$
                This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
                $endgroup$
                – Discrete lizard
                13 hours ago
















              $begingroup$
              This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
              $endgroup$
              – Discrete lizard
              13 hours ago




              $begingroup$
              This does explain why computers could be surprising which is the first half of the quote, but you do not address the part of the quote that explains why a particular argument that machines cannot surprise is fallacious.
              $endgroup$
              – Discrete lizard
              13 hours ago











              3












              $begingroup$

              This is the idea of emergence, which is when complex behavior of a result results from the interaction of relatively simple rules. There are lots of examples of this in nature, as that link points out. Insect colonies, bird flocks, schools of fish, and of course, consciousness. In a flock of birds or school of fish, each individual in the swam is only making decisions based on the others immediately surrounding them, but when you put a bunch of those individuals together all following those rules, you start to see more coordinated behavior than you'd expect without a higher level plan. If you go on Youtube and watch demonstrations of robot swarms, you see that they all avoid hitting each other and work in unison. Surprisingly this doesn't need to be accomplished by having a single central computer coordinate the behavior of each individual robot but can instead be done using swarm robotics where, like the insects or the birds or the fish, each robot is making local decisions which leads to emergent coordination.



              Another interesting demonstration of emergent behavior is Conway's Game of Life. The rules for the game are extremely simple, but can lead to very fascinating results



              A tempting argument against the ability of computers to gain human-intelligence is to say that since they can only do precisely what they're programmed to do, they must only exhibit the intelligence that we program them with. If this were true, then we would also not expect the relatively simple behavior of neurons to give rise to human intelligence. Yet as far as we can tell, this IS the case and consciousness is an emergent property of neural processing. I'm sure Turing would have loved to see what's become possible today with the use of artificial neural networks






              share|cite|improve this answer










              New contributor




              mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
              Check out our Code of Conduct.






              $endgroup$


















                3












                $begingroup$

                This is the idea of emergence, which is when complex behavior of a result results from the interaction of relatively simple rules. There are lots of examples of this in nature, as that link points out. Insect colonies, bird flocks, schools of fish, and of course, consciousness. In a flock of birds or school of fish, each individual in the swam is only making decisions based on the others immediately surrounding them, but when you put a bunch of those individuals together all following those rules, you start to see more coordinated behavior than you'd expect without a higher level plan. If you go on Youtube and watch demonstrations of robot swarms, you see that they all avoid hitting each other and work in unison. Surprisingly this doesn't need to be accomplished by having a single central computer coordinate the behavior of each individual robot but can instead be done using swarm robotics where, like the insects or the birds or the fish, each robot is making local decisions which leads to emergent coordination.



                Another interesting demonstration of emergent behavior is Conway's Game of Life. The rules for the game are extremely simple, but can lead to very fascinating results



                A tempting argument against the ability of computers to gain human-intelligence is to say that since they can only do precisely what they're programmed to do, they must only exhibit the intelligence that we program them with. If this were true, then we would also not expect the relatively simple behavior of neurons to give rise to human intelligence. Yet as far as we can tell, this IS the case and consciousness is an emergent property of neural processing. I'm sure Turing would have loved to see what's become possible today with the use of artificial neural networks






                share|cite|improve this answer










                New contributor




                mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.






                $endgroup$
















                  3












                  3








                  3





                  $begingroup$

                  This is the idea of emergence, which is when complex behavior of a result results from the interaction of relatively simple rules. There are lots of examples of this in nature, as that link points out. Insect colonies, bird flocks, schools of fish, and of course, consciousness. In a flock of birds or school of fish, each individual in the swam is only making decisions based on the others immediately surrounding them, but when you put a bunch of those individuals together all following those rules, you start to see more coordinated behavior than you'd expect without a higher level plan. If you go on Youtube and watch demonstrations of robot swarms, you see that they all avoid hitting each other and work in unison. Surprisingly this doesn't need to be accomplished by having a single central computer coordinate the behavior of each individual robot but can instead be done using swarm robotics where, like the insects or the birds or the fish, each robot is making local decisions which leads to emergent coordination.



                  Another interesting demonstration of emergent behavior is Conway's Game of Life. The rules for the game are extremely simple, but can lead to very fascinating results



                  A tempting argument against the ability of computers to gain human-intelligence is to say that since they can only do precisely what they're programmed to do, they must only exhibit the intelligence that we program them with. If this were true, then we would also not expect the relatively simple behavior of neurons to give rise to human intelligence. Yet as far as we can tell, this IS the case and consciousness is an emergent property of neural processing. I'm sure Turing would have loved to see what's become possible today with the use of artificial neural networks






                  share|cite|improve this answer










                  New contributor




                  mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.






                  $endgroup$



                  This is the idea of emergence, which is when complex behavior of a result results from the interaction of relatively simple rules. There are lots of examples of this in nature, as that link points out. Insect colonies, bird flocks, schools of fish, and of course, consciousness. In a flock of birds or school of fish, each individual in the swam is only making decisions based on the others immediately surrounding them, but when you put a bunch of those individuals together all following those rules, you start to see more coordinated behavior than you'd expect without a higher level plan. If you go on Youtube and watch demonstrations of robot swarms, you see that they all avoid hitting each other and work in unison. Surprisingly this doesn't need to be accomplished by having a single central computer coordinate the behavior of each individual robot but can instead be done using swarm robotics where, like the insects or the birds or the fish, each robot is making local decisions which leads to emergent coordination.



                  Another interesting demonstration of emergent behavior is Conway's Game of Life. The rules for the game are extremely simple, but can lead to very fascinating results



                  A tempting argument against the ability of computers to gain human-intelligence is to say that since they can only do precisely what they're programmed to do, they must only exhibit the intelligence that we program them with. If this were true, then we would also not expect the relatively simple behavior of neurons to give rise to human intelligence. Yet as far as we can tell, this IS the case and consciousness is an emergent property of neural processing. I'm sure Turing would have loved to see what's become possible today with the use of artificial neural networks







                  share|cite|improve this answer










                  New contributor




                  mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.









                  share|cite|improve this answer



                  share|cite|improve this answer








                  edited 11 hours ago





















                  New contributor




                  mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.









                  answered 11 hours ago









                  mowwwalkermowwwalker

                  1312




                  1312




                  New contributor




                  mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.





                  New contributor





                  mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.






                  mowwwalker is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.























                      -2












                      $begingroup$

                      Surprise is something you did not expect. Turing says once you see what the machine says, you may be surprised.






                      share|cite|improve this answer








                      New contributor




                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.






                      $endgroup$









                      • 3




                        $begingroup$
                        This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
                        $endgroup$
                        – Discrete lizard
                        13 hours ago
















                      -2












                      $begingroup$

                      Surprise is something you did not expect. Turing says once you see what the machine says, you may be surprised.






                      share|cite|improve this answer








                      New contributor




                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.






                      $endgroup$









                      • 3




                        $begingroup$
                        This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
                        $endgroup$
                        – Discrete lizard
                        13 hours ago














                      -2












                      -2








                      -2





                      $begingroup$

                      Surprise is something you did not expect. Turing says once you see what the machine says, you may be surprised.






                      share|cite|improve this answer








                      New contributor




                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.






                      $endgroup$



                      Surprise is something you did not expect. Turing says once you see what the machine says, you may be surprised.







                      share|cite|improve this answer








                      New contributor




                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.









                      share|cite|improve this answer



                      share|cite|improve this answer






                      New contributor




                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.









                      answered 14 hours ago









                      David ReichardDavid Reichard

                      1




                      1




                      New contributor




                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.





                      New contributor





                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.






                      David Reichard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                      Check out our Code of Conduct.








                      • 3




                        $begingroup$
                        This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
                        $endgroup$
                        – Discrete lizard
                        13 hours ago














                      • 3




                        $begingroup$
                        This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
                        $endgroup$
                        – Discrete lizard
                        13 hours ago








                      3




                      3




                      $begingroup$
                      This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
                      $endgroup$
                      – Discrete lizard
                      13 hours ago




                      $begingroup$
                      This is the first half of the quote, but probably not the part that may be confusing. And that machines being capable of surprise is not exactly what Turing claims here, he merely claims that a particular argument that machines cannot surprise is fallacious.
                      $endgroup$
                      – Discrete lizard
                      13 hours ago










                      smwikipedia is a new contributor. Be nice, and check out our Code of Conduct.










                      draft saved

                      draft discarded


















                      smwikipedia is a new contributor. Be nice, and check out our Code of Conduct.













                      smwikipedia is a new contributor. Be nice, and check out our Code of Conduct.












                      smwikipedia is a new contributor. Be nice, and check out our Code of Conduct.
















                      Thanks for contributing an answer to Computer Science Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      Use MathJax to format equations. MathJax reference.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcs.stackexchange.com%2fquestions%2f107198%2fwhat-does-turing-mean-by-this-statement%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      數位音樂下載

                      When can things happen in Etherscan, such as the picture below?

                      格利澤436b