Using LinEst() on data with gaps

A comment on Using LINEST for non-linear curve fitting asked if it was possible to use the function on data with gaps.  There are several options:

  1. Copy and sort the data to remove the gaps.
  2. Use the chart trendline function
  3. Use one of the on-sheet functions provided by Lori Miller in a reply to the comment.
  4. Use the LinEstgap() User Defined Function (UDF) that has been added to the Linest-poly spreadsheet.

A copy of linest-poly, including full open source code, may be downloaded from: LinEst-Poly.xls.  Both the on-sheet functions and the LinEstGap() function must be entered as an array function, as described here: Using Array Formulas

Use of these options (other than number 1) is shown in the screen-shots below:

With continuous data all options give the same result:

LinEst with continuous data

Deleting some data, the LinEst function returns an error, but the other functions return a result with the rows with blank cells ignored.  This result is consistent with the chart trend line result.

Linest on data with gaps

Using linest on filtered data (but with data in every row) includes the hidden rows in the analysis, as does the simpler of the two on-sheet functions.  The other options use only the visible data:

Linest with filtered data

 

The on-sheet formulas treat cells containing 0 (zero) as being blank, whereas the other options treat 0 as a valid data value:

LinEst including x value of zero

Advertisements
This entry was posted in Arrays, Excel, UDFs, VBA and tagged , , , , . Bookmark the permalink.

25 Responses to Using LinEst() on data with gaps

  1. Lori Miller says:

    Doug – The formula is just using the fact that regression coefficients are unchanged by inserting rows of zeroes into the regressor matrix. Good spot on the handling of 0’s, I think a simplified version that also accounts for this is:
    =LINEST(1*Y,ISNUMBER(Y)*(X+1E-99)^{0,1,2},0)

    (adding 1E-99 avoids 0^0, IFERROR could be used instead). I also like the UDF, perhaps one could include a generic function as well so that other functions like Alglib could work with gap data. Something like:
    =LINEST(Gap(Y,X),Gap(X,Y)^{1,2))

    PS I’m not sure the latest version of the file is linked so perhaps the revised version be included instead? Thanks.

    Like

    • dougaj4 says:

      PS I’m not sure the latest version of the file is linked so perhaps the revised version be included instead? Thanks

      Damned case-sensitive Unix file servers!

      I’ve linked to the right one now – thanks for letting me know.

      Like

    • Ed Pink says:

      Hi Lori,

      your simplified version looks great, i copied the 1E-99 version and confirmed that it worked… but i played around a bit with ISERROR and couldnt get it to work (it was always gives the same result as the other on-sheet functions, ignoring the X_4 = 0 data points). How would the complete formula look with ISERROR ?

      what i tried:
      =LINEST(1*Y_4,ISNUMBER(Y_4)*ISERROR((X_4)^{0,1,2},0);0)

      Thanks
      Ed

      Like

      • Lori Miller says:

        Ed – try with: IFERROR((X_4)^{0,1,2},1). i.e. set 0^0 = 1. A related formula for a weighted regression is here: http://stackoverflow.com/questions/11087773/weighted-trendline

        Like

      • Ed Pink says:

        Hallo Lori
        oh yeah.. thats right, 0^0=1 …. woops. guess im gonna go back to school.. i knew when i burnt my books after finishing high school that it would come back to bite me one day!

        Thanks a heap ! Ed

        Like

      • Lori Miller says:

        0^0 is actually an interesting topic in itself. In high school it is taken to be indeterminate since 0^x and x^0 give different values as x->0. Setting the value equal to 1 is a common convention in further study and simplifies expressions like the binomial theorem but most computer compilers throw an error for this expression. Much more discussion can be found in online forums.

        Like

  2. dougaj4 says:

    Lori – good idea on the generic gap function – I’ll put it on th e”to do” list 🙂

    Like

  3. jp says:

    Doug … Hi … Great posts/comments on LINEST/LOGEST … just wondering if I can quickly adapt the LINEST GAP function for exponential curves? …. cheers, James

    Like

  4. dougaj4 says:

    Hi James – I’ll need to remind myself of the details. I’ll try and have a look at it over the weekend (if I don’t get distracted by other things :))

    Like

  5. Mike says:

    Doug, I tried to modify your UDF code to suit the case that the arrays are in rows rather than columns….but unsuccessfully. It’s for a second order polynomial. Can you help?
    Thanks in advance…

    Like

  6. Pingback: Linestgap with data in rows | Newton Excel Bach, not (just) an Excel Blog

  7. Pingback: Daily Download 14: Curve Fitting 1 | Newton Excel Bach, not (just) an Excel Blog

  8. Yux Yux says:

    Thankful for this post and the comments. Helped me get past my little obstacle.
    Just thinking about the problem of “The on-sheet formulas treat cells containing 0 (zero) as being blank, whereas the other options treat 0 as a valid data value:”
    Can’t combining onsheet formula 2 with Lori’s zero value solution can get one past that little issue?:
    =LINEST( SUBTOTAL(3,OFFSET(X_4,ROW(X_4)-MIN(ROW(X_4)),,1)) *Y_4, IF(X_4″”,SUBTOTAL(3,OFFSET(Y_4,ROW(Y_4)-MIN(ROW(Y_4)),,1))*(X_4+1E-99)^{0,1,2},0),0)

    Like

  9. Yux Yux says:

    That was supposed to be:
    =LINEST( SUBTOTAL(3,OFFSET(X_4,ROW(X_4)-MIN(ROW(X_4)),,1)) *Y_4,
    IF(X_4″”,SUBTOTAL(3,OFFSET(Y_4,ROW(Y_4)-MIN(ROW(Y_4)),,1))*(X_4+1E-99)^{0,1,2},0),0)

    Like

  10. Yux Yux says:

    One more try with the “Not Equal To” sign after IF(X_4 that is not supposed to be tags…

    =LINEST( SUBTOTAL(3,OFFSET(X_4,ROW(X_4)-MIN(ROW(X_4)),,1)) *Y_4,
    IF(X_4 < > “”,SUBTOTAL(3,OFFSET(Y_4,ROW(Y_4)-MIN(ROW(Y_4)),,1))*(X_4+1E-99)^{0,1,2},0),0)

    Like

  11. Aleks says:

    Hi,

    This code is exactly what I’ve been looking for. I am trying to get the Linestgap function in a spreadsheet I am working on. Move or copy -> create copy in my current worksheet doesn’t work. Could someone please advise?

    Many Thanks,

    Aleks.

    Like

  12. Aleks says:

    Also,

    Can anyone give examples of using linest gap for the other common functions?

    How can LinEstGap be used to determine r-squared values?

    Cheers!

    Like

  13. Rod says:

    Many thanks for the code. It works wonderfully and has saved me a lot of time in spreadsheeting processing.

    Just a quick question…if instead of blanks cells I had #NA, what would have to change in the code for it to be used ?

    Like

    • dougaj4 says:

      It actually allows that already. If you look at the examples on the Data with gaps sheet, change cell D34 to:
      =LINESTgap(B31:L31,B28:L28^{1;2;3},,,TRUE)
      You will be able to change some of the data to =NA() (or another error function), and that data will be ignored, rather than returning an error.
      If you open the function in the function wizard, you will see that the last argument is “IgnoreErrors”. The default is False.

      Like

  14. Mike says:

    Hello,

    I would like to transpose the Linestgap-function from cells D20:F20 to cells G20:G22.
    Although whatever i try, i do not succeed. Is there a way to do this?
    It seems it is due to the array-format of the formula. Although there should be a method to enter the formula vertically instead of horizontally i suppose?

    Many thanks!
    Mike

    Like

    • dougaj4 says:

      You can wrap the function with the Transpose function:
      =TRANSPOSE(LINESTgap(C4:C14,A4:A14^{1,2}))
      Enter the function as above, then select three rows in a single column (with the function In the top row), press F2, then Ctrl-Shift-Enter

      Liked by 1 person

  15. Mike says:

    Thanks a lot dougaj4, this works!

    Like

  16. Julian says:

    Hello,

    I would like to run a multiple (!) factor analysis through linest, again excluding all rows that contain zeros (or if that’s easier blank cells). Is there a way I can do this using the above shown linest functions?

    FYI – my Y variables are located at C15:C26, my X1 variables are located at D15:D26, and my X2 variables are located at E15:E26.

    Thanks in advance!
    Julian

    Like

    • dougaj4 says:

      The example shown (and in the download file) has the x data in two columns. In the example the second column is the square of the first column, so it fits a quadratic function to the data, but it works the same if the two columns are independent variables, in which case you get the coefficients for y = ax1 + bx2 + c.

      Also the x range can be as many columns as you want. It’s not limited to one or two.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s