Variance of Monte Carlo integration with importance sampling





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







3












$begingroup$


I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_{0}^{1} e^{x}dx$. For the importance sampling version, I rewrite $int_{0}^{1} e^{x}dx = int_{0}^{1} e^{x}/p(x)cdot p(x)dx$ where $p(x) = 2.5x^{1.5}$. Then



$$hat{I} = frac{1}{N}sum_{j=1}^{N} frac{f(x_{j})}{p(x_{j})},$$



where $x_{j}$ are sampled from $p(x_{j})$ (I use an inverse transform method here). For the variance, I have $sigma_{I}^{2} = hat{sigma}_{I}^{2}/N$ and



$$hat{sigma}_{I}^{2} = frac{1}{N} sum_{j=1}^{N} frac{f(x_{j})^{2}}{g(x_{j})^{2}} - hat{I}^{2}.$$



I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.



enter image description here






monte-carlo integral importance-sampling







share|cite|improve this question









$endgroup$



















    3












    $begingroup$


    I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_{0}^{1} e^{x}dx$. For the importance sampling version, I rewrite $int_{0}^{1} e^{x}dx = int_{0}^{1} e^{x}/p(x)cdot p(x)dx$ where $p(x) = 2.5x^{1.5}$. Then



    $$hat{I} = frac{1}{N}sum_{j=1}^{N} frac{f(x_{j})}{p(x_{j})},$$



    where $x_{j}$ are sampled from $p(x_{j})$ (I use an inverse transform method here). For the variance, I have $sigma_{I}^{2} = hat{sigma}_{I}^{2}/N$ and



    $$hat{sigma}_{I}^{2} = frac{1}{N} sum_{j=1}^{N} frac{f(x_{j})^{2}}{g(x_{j})^{2}} - hat{I}^{2}.$$



    I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.



    enter image description here






    monte-carlo integral importance-sampling







    share|cite|improve this question









    $endgroup$















      3












      3








      3


      3



      $begingroup$


      I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_{0}^{1} e^{x}dx$. For the importance sampling version, I rewrite $int_{0}^{1} e^{x}dx = int_{0}^{1} e^{x}/p(x)cdot p(x)dx$ where $p(x) = 2.5x^{1.5}$. Then



      $$hat{I} = frac{1}{N}sum_{j=1}^{N} frac{f(x_{j})}{p(x_{j})},$$



      where $x_{j}$ are sampled from $p(x_{j})$ (I use an inverse transform method here). For the variance, I have $sigma_{I}^{2} = hat{sigma}_{I}^{2}/N$ and



      $$hat{sigma}_{I}^{2} = frac{1}{N} sum_{j=1}^{N} frac{f(x_{j})^{2}}{g(x_{j})^{2}} - hat{I}^{2}.$$



      I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.



      enter image description here






      monte-carlo integral importance-sampling







      share|cite|improve this question









      $endgroup$




      I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_{0}^{1} e^{x}dx$. For the importance sampling version, I rewrite $int_{0}^{1} e^{x}dx = int_{0}^{1} e^{x}/p(x)cdot p(x)dx$ where $p(x) = 2.5x^{1.5}$. Then



      $$hat{I} = frac{1}{N}sum_{j=1}^{N} frac{f(x_{j})}{p(x_{j})},$$



      where $x_{j}$ are sampled from $p(x_{j})$ (I use an inverse transform method here). For the variance, I have $sigma_{I}^{2} = hat{sigma}_{I}^{2}/N$ and



      $$hat{sigma}_{I}^{2} = frac{1}{N} sum_{j=1}^{N} frac{f(x_{j})^{2}}{g(x_{j})^{2}} - hat{I}^{2}.$$



      I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.



      enter image description here






      monte-carlo integral importance-sampling




      monte-carlo integral importance-sampling






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked Apr 1 at 18:26









      user1799323user1799323

      1234




      1234






















          1 Answer
          1






          active

          oldest

          votes


















          3












          $begingroup$

          enter image description hereThis is a good illustration of the dangers of importance sampling: while
          $$int_0^1 frac{e^x}{p(x)}, p(x)text{d} x = int_0^1 e^x text{d} x = I$$
          shows that $hat{I}_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
          $$int_0^1 left(frac{e^x}{p(x)}right)^2, p(x)text{d} x = int_0^1 frac{e^{2x}}{2.5 x^{1.5}} text{d} x = infty$$
          since the integral diverges in $x=0$. For instance,



          > x=runif(1e7)^{1/2.5}
          
> range(exp(x)/x^{1.5})
          
[1]     2.718282 83403.685972


          shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since



           > mean(exp(x)/x^{1.5})/2.5
          
 [1] 1.717576
          
 > var(exp(x)/x^{1.5})/(2.5)^2/1e7
          
 [1] 2.070953e-06


          but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^{-1/2}$.)






          share|cite|improve this answer











          $endgroup$














            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "65"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400628%2fvariance-of-monte-carlo-integration-with-importance-sampling%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            3












            $begingroup$

            enter image description hereThis is a good illustration of the dangers of importance sampling: while
            $$int_0^1 frac{e^x}{p(x)}, p(x)text{d} x = int_0^1 e^x text{d} x = I$$
            shows that $hat{I}_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
            $$int_0^1 left(frac{e^x}{p(x)}right)^2, p(x)text{d} x = int_0^1 frac{e^{2x}}{2.5 x^{1.5}} text{d} x = infty$$
            since the integral diverges in $x=0$. For instance,



            > x=runif(1e7)^{1/2.5}
            
> range(exp(x)/x^{1.5})
            
[1]     2.718282 83403.685972


            shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since



             > mean(exp(x)/x^{1.5})/2.5
            
 [1] 1.717576
            
 > var(exp(x)/x^{1.5})/(2.5)^2/1e7
            
 [1] 2.070953e-06


            but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^{-1/2}$.)






            share|cite|improve this answer











            $endgroup$


















              3












              $begingroup$

              enter image description hereThis is a good illustration of the dangers of importance sampling: while
              $$int_0^1 frac{e^x}{p(x)}, p(x)text{d} x = int_0^1 e^x text{d} x = I$$
              shows that $hat{I}_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
              $$int_0^1 left(frac{e^x}{p(x)}right)^2, p(x)text{d} x = int_0^1 frac{e^{2x}}{2.5 x^{1.5}} text{d} x = infty$$
              since the integral diverges in $x=0$. For instance,



              > x=runif(1e7)^{1/2.5}
              
> range(exp(x)/x^{1.5})
              
[1]     2.718282 83403.685972


              shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since



               > mean(exp(x)/x^{1.5})/2.5
              
 [1] 1.717576
              
 > var(exp(x)/x^{1.5})/(2.5)^2/1e7
              
 [1] 2.070953e-06


              but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^{-1/2}$.)






              share|cite|improve this answer











              $endgroup$
















                3












                3








                3





                $begingroup$

                enter image description hereThis is a good illustration of the dangers of importance sampling: while
                $$int_0^1 frac{e^x}{p(x)}, p(x)text{d} x = int_0^1 e^x text{d} x = I$$
                shows that $hat{I}_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
                $$int_0^1 left(frac{e^x}{p(x)}right)^2, p(x)text{d} x = int_0^1 frac{e^{2x}}{2.5 x^{1.5}} text{d} x = infty$$
                since the integral diverges in $x=0$. For instance,



                > x=runif(1e7)^{1/2.5}
                
> range(exp(x)/x^{1.5})
                
[1]     2.718282 83403.685972


                shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since



                 > mean(exp(x)/x^{1.5})/2.5
                
 [1] 1.717576
                
 > var(exp(x)/x^{1.5})/(2.5)^2/1e7
                
 [1] 2.070953e-06


                but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^{-1/2}$.)






                share|cite|improve this answer











                $endgroup$



                enter image description hereThis is a good illustration of the dangers of importance sampling: while
                $$int_0^1 frac{e^x}{p(x)}, p(x)text{d} x = int_0^1 e^x text{d} x = I$$
                shows that $hat{I}_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
                $$int_0^1 left(frac{e^x}{p(x)}right)^2, p(x)text{d} x = int_0^1 frac{e^{2x}}{2.5 x^{1.5}} text{d} x = infty$$
                since the integral diverges in $x=0$. For instance,



                > x=runif(1e7)^{1/2.5}
                
> range(exp(x)/x^{1.5})
                
[1]     2.718282 83403.685972


                shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since



                 > mean(exp(x)/x^{1.5})/2.5
                
 [1] 1.717576
                
 > var(exp(x)/x^{1.5})/(2.5)^2/1e7
                
 [1] 2.070953e-06


                but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^{-1/2}$.)







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited Apr 1 at 19:02

























                answered Apr 1 at 18:44









                Xi'anXi'an

                59.1k897365




                59.1k897365






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Cross Validated!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400628%2fvariance-of-monte-carlo-integration-with-importance-sampling%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    數位音樂下載

                    Image
                    數位音樂下載 為非以「實體」方式來販賣音樂的相關產品，而使用數位格式如mp3、AAC等文件格式來進行銷售販賣的一种音乐产业形式。著名的數位音樂下載服務商有iTunes Store、RecoChoku、Google Play音乐和微软Zune等。数字音乐下载在二十一世纪初一度风靡全球， [1] [2] 使得传统唱片产业走向衰落，但其自身也在2010年代流媒体音乐服务如Apple Music、Spotify等的挤压下迅速失去市场份额。 [3] [4] 数字音乐下载有便于携带、传播迅速、制作简单成本低等优点，与流媒体音乐相比也能更好保障音乐人权益； [5] 但它同时很可能带来侵犯版权和滥用的情形， [6] 也给音乐产业带来了诸多不利影响。数字音乐下载为音乐人和乐迷提供了更加便捷直接的交流渠道，也一定程度上推动了网络音乐人的出现和独立音乐的迅速发展。 [7] 目录 1 历史 1.1 技术积淀（1987-2000） 1.2 出现与兴起（2000-2004） 1.3 唱片市场的颠覆者（2004-2007） 1.4 巅峰和衰落（2005-2013） 1.5 摇摇欲坠的现状与不可预知的明天（2014至今） 2 主要文件格式 2.1 PCM编码格式 2.1.1 WAV 2.1.2 APE 2.1.3 FLAC 2.2 苹果推出的无损文件格式 2.2.1 AIFF 2.2.2 ALAC 2.3 有损压缩格式 2.3.1 MP3 2.3.2 AAC 3 主要市场 3.1 iTunes Store 3.2 Google Play音乐商店 3.3 亚马逊音乐商店 3.4 部分其他音乐商店 4 影响与评价 4.1 对音乐产业的影响 4.2 对科技行业的影响 4.3 正面评价 4.4 负面评价 4.4.1 侵权问题 4.4.2 经营模式不明晰 4.4.3 不适应移动时代 4.5 文化现象 5 延伸阅读 6 参考资料 历史 苹果iTunes+iPod的软硬件搭配让数字音...
                    Read more

                    When can things happen in Etherscan, such as the picture below?

                    Image
                    2 I don't know why balance of the address is 0. when can this situation? etherscan balances share | improve this question asked yesterday Ru Hi Ru Hi 11 1 New contributor Ru Hi is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. add a comment  |  ...
                    Read more

                    格利澤436b

                    Image
                    格利泽436b 太陽系外行星 太陽系外行星列表 艺术家笔下的格利泽436b 母恆星 母恆星 Gliese 436 星座 獅子座 赤經 ( α ) 11 h  42 m  11.0941 s [1] 赤緯 ( δ ) +26° 42′ 23.652″ [1] 距離 33.4 ly (10.2 pc) 光譜類型 M2.5 V [1] 軌道參數 半长轴 ( a ) 0.0291±0.0004 [2] AU 軌道離心率 ( e ) 0.150±0.012 [2] 公轉週期 ( P ) 2.643904±0.000005 [3] d (0.00723849 y) 軌道傾角 ( i ) 85.8 +0.21 −0.25 [3] ° 角距 ( θ ) 2.794 mas 近星點時間 ( T 0 ) 2,451,551.716 ±0.01 JD 半振幅 ( K ) 18.68±0.8 m/s 物理性质 质量 ( m ) 22.2±1.0 [2] M ⊕ 半徑 ( r ) 4.327±0.183 [2] [4] R ⊕ 密度 ( ρ ) 1.51 g cm -3 表面重力 ( g ) 1.18 g 溫度 ( T ) 712±36 [2] K 發現 發現時間 2004 發現者 巴特勒、沃格特、 馬西 et al. 發現方法 徑向速度、凌日法 發現地點 美國加利福尼亞州 發表論文 出版 其他名稱 Ross 905 b, GJ 436 b, [5] LTT 13213 b, GCTP 2704.10 b, LHS 310, AC+27:28217 b, Vyssotsky 616 b, HIP 57087 b, GEN# +9.80120068 b, LP 319-75 b, G 121-7 b, LSPM J1142+2642 b, 1RXS J114211.9+264328 b, ASCC 683818 b, G 147-68 b, UCAC2 41198281 b, BP...
                    Read more