Image source: The Motley Fool.
Byline Bancorp, Inc. (BY 3.19%)
Q4Â 2019 Earnings Call
Jan 24, 2020, 10:00 a.m. ET
Contents:
- Prepared Remarks
- Questions and Answers
- Call Participants
Prepared Remarks:
Operator
Good day, and welcome to the Fourth Quarter 2019 Byline Bancorp Incorporated Earnings Conference Call. [Operator Instructions] I would now like to turn the conference over to Tony Rossi of Financial Profiles. Please go ahead.
Tony Rossi -- Analyst
Thank you, Jason. Good morning everyone and thank you for joining us today for the Byline Bancorp fourth quarter 2019 earnings call. We will be using a slide presentation as part of our discussion this morning. Please visit the Events and Presentations page of Byline's Investor Relations website for access to the presentation.
Before we begin, I'd like to remind you that this conference call contains forward-looking statements with respect to the future performance and financial condition of Byline Bancorp that involve risks and uncertainties. Various factors could cause actual results to be materially different from any future results expressed or implied by such forward-looking statements. These factors are discussed in the Company's SEC filings, which are available on the Company's website. The Company disclaims any obligation to update any forward-looking statements made during the call.
Management may refer to non-GAAP measures, which are intended to supplement, but not substitute for the most directly comparable GAAP measures. The press release available on the website contains the financial and other quantitative information to be discussed today as well as the reconciliation of the GAAP to non-GAAP measures.
With that, I'd like to turn the call over to Alberto Paracchini President and CEO. Alberto?
Alberto J. Paracchini -- President & Chief Executive Officer
Thank you, Tony. Good morning and welcome everyone to our fourth quarter earnings call. We appreciate all of you joining us this morning. With me here today is Lindsay Corby, our CFO; and Owen Beacom, our Chief Credit Officer. As we normally do, I will provide you with the highlights for the year and quarter, and then pass it over to Lindsay who will walk you through our results in more detail. I'll come back at the end with some closing remarks before opening the call up for questions. As a reminder, you can follow our comments with the help of the presentation you can find in the Investor Relations section of our website.
2019 for us to be a busy year for Byline. We completed an important system conversion at the start of the year that finished the integration of First Evanston, closed and integrated Oak Park River Forest Bankshares, executed against our organic growth strategies, capitalized on market opportunities and delivered solid financial results with EPS for the year coming in at $1.48, up 25% on a year-over-year basis and total assets exceeding $5.5 billion. Our profitability continued to improve with both ROAA and ROATCE increasing over 2018, and our adjusted efficiency ratio also improving on a year-over-year basis. Given our improving profitability, increase in tangible book value per share and strong capital levels, we announced both a share repurchase program and declared a cash dividend for the quarter to allow us to manage capital levels, while continuing to focus on organic growth and acquisition opportunities.
In terms of highlights for the quarter, earnings came in at $15.9 million or $0.41 per diluted share. Adjusting for charges related to merger, system conversions and other asset impairment costs, earnings came in at $16.1 million or $0.42 per diluted share. Profitability remained strong with ROAA and ROATCE on an adjusted basis of 117 basis points and 12.4% respectively.
Total assets increased by 1.5% sequentially, while loans declined slightly during the quarter despite very solid loan originations of $179 million, which were up from $97 million last quarter. Consistent with the third quarter, our originations were stronger in the commercial small business and sponsor finance groups. We continue to see good deal flow across the business, but believe given the stage of the economy and credit cycle, it's important to remain disciplined and selective when evaluating opportunities.
Payoff and paydown activity was elevated at $190 million, and offset the strong originations we saw during the quarter. Our CRE and residential portfolios were particularly impacted. On the CRE front, we saw a number of projects reach completion and borrowers opting to sell their projects or secure long-term permanent financing. Our residential portfolio saw increased paydown stemming from the lower rate environment. Our commercial portfolio, including small business, also saw higher-than-anticipated payoffs coming from business sales or refinancings at terms outside of our credit appetite.
On the deposit front, we had a strong quarter of deposit growth, particularly on non-interest bearing and other core categories. Time deposits declined to 28.3% of total deposits and the shift in mix contributed to deposit costs declining 6 basis points for the quarter. Our basic strategy of pursuing full banking relationships and doing more business with existing clients continues to generate very good results. Given the deposit and loan growth dynamics during the quarter, we saw our loan-to-deposit ratio ticked down to 91.5%, and our liquidity in the form of investment securities increase. We expect to redeploy that excess liquidity over time in our loan portfolio.
Revenues for the quarter declined as expected given the current rate environment, but were up 1.2% over last year. Net interest income was lower, impacted by a lower margin compared to the third quarter. Non-interest income was down slightly, driven by lower gain on sale revenue due primarily to lower average premiums as well as a fair value charge taken against our servicing asset.
On the expense side, we saw operating expenses declined sequentially by $1.8 million from the third quarter. Lastly, asset quality improved with higher resolution activity driving lower non-performing loan levels. Provision expense declined and covered lower net charge-offs, which came down by 14 basis points from last quarter to 42 basis points in the fourth quarter. The allowance increased 2 basis points to 84 basis points at the end of the year.
With that, I'd like to turn over the call to Lindsay who will provide you more detail on our results.
Lindsay Y. Corby -- Chief Financial Officer
Thanks, Alberto. Good morning, everyone. Starting with loans and leases. Our total loans and leases were $3.8 billion at December 31, a net decrease of $45.4 million from the prior quarter. The decrease in total loans and leases was primarily due to a higher level of payoffs and paydowns in the quarter. Payoffs came in at $190 million, compared to $150 million in the third quarter. Our originated loan portfolio increased approximately $68 million net for the quarter. The growth was primarily driven by our commercial and commercial real estate portfolios. The growth in the originated loan portfolio was offset by $130 million decrease in our acquired portfolio. Approximately half of the decrease in the acquired portfolio related to the natural movement to originated and the remainder stems from paydowns and payoffs on loans and relationships, not considered core to our business, are lower graded credits that we believe did not justify the risk-adjusted pricing and structure offered by the market.
As Alberto mentioned earlier, most of our new loan production continues to come in the C&I and small business lending areas, which has helped to improve the balanced diversification in our portfolio. Over the past year, CRE loans declined 34% of our total loan portfolio, down from 36% at the end of 2018, while C&I loans increased 2 percentage points to 35% over the same time period. During the fourth quarter of 2019, we moved up to the Number 4 SBA lender nationwide and continue to be ranked Number 1 in Illinois and Wisconsin. We had a very productive quarter, closing $132 million of loan commitments, up from $125 million in the third quarter. With the strong production, our managed government guaranteed portfolio increased by $28 million in the fourth quarter to just under $1.9 billion.
Moving on to deposits. We had another strong quarter of core deposit growth, with our total deposits increasing $67.3 million to $4.1 billion at December 31. The growth was entirely driven by increases in our lower cost deposit categories, most notably, non-interest bearing and money market balances. The growth in these categories is largely being driven by inflows of commercial deposits. On an average basis, our non-interest-bearing deposits were $65 million higher than the previous quarter. The growth in these lower cost areas allowed us to run off $97 million in time deposits, improving our overall deposit mix. As a result of the improved deposit mix, both our total cost of deposits and our cost of interest-bearing deposits decreased 6 basis points from the prior quarter, making a significant inflection point in our ability to manage our funding costs.
We continue to be well positioned from a liability standpoint. During the fourth quarter, we began to see higher cost funding repricing at levels below their current rates of maturity. So assuming no change in the outlook for rates and market conditions as our time deposits continue to renew, we expect to see a continued decline in the cost of these deposits during the first half of 2020.
Moving on to net interest income and margin. Our net interest income decreased $3.9 million from the third quarter as a result of lower accretion and the lower rate environment. Our net interest margin was 4.32% in the fourth quarter, down 30 basis points from last quarter. Accretion income on acquired loans contributed 43 basis points to the margin in the fourth quarter, down from 62 basis points in the last quarter. Excluding accretion income, our net interest margin was 3.89%. The 11 basis point decrease from the previous quarter was primarily due to the average loan and lease yield, excluding accretion income, declining to 5.45% [Phonetic] from 5.67% [Phonetic]. Approximately 50% of our portfolio is floating rate and split between LIBOR and Prime. The decrease in the average loan yields, excluding accretion income, was due to the impact of the September and October rate cut. The decline in our average loan yields offset the decrease that we saw in our cost of deposit.
With the repricing in our loan portfolio largely completed and the continued repricing of our higher cost funding at lower rates, we believe we are in a good position to maintain our net interest margin, excluding accretion income subject to no additional changes in the Fed funds rate and the pace of loan growth.
Turning to non-interest income on Slide 8. In the third quarter, our non-interest income decreased by $290,000 from the prior quarter. The decrease was primarily due to a $2.5 million fair value adjustment on our servicing asset to reflect increased prepayment fees and discount rate, up from $1.6 million adjustment last quarter. During the fourth quarter, we sold $101.5 million of government guaranteed loans compared with $93.3 million of loans sold in the prior quarter. However, net average premiums received during the quarter decreased 81 basis points to 10.60% resulting in a decline of $670,000 in our net gain on government guaranteed loans. Offsetting the volatility of the servicing asset and average premium decreases, we saw improved fee income on our deposits, increased interchange income and an increase in swap revenues.
Looking at our non-interest expense. Our fourth quarter expenses included $286,000 of merger-related expense, core system conversion expense and impairment charges on assets held for sale. Adjusting for these items in both periods, our non-interest expense decreased $1.8 million from the prior quarter. The decrease was primarily due to the full quarter impact of the additional cost savings resulting from the Oak Park River Forest integration and system conversion.
From the expense perspective, we will continue to be disciplined and focused in our management of expenses, so that we can realize additional operating leverage as our revenue increases. As we mentioned a few months ago during the first quarter, we are consolidating four branches that will result in $1.2 million in annualized cost savings that should help our efforts to manage expense levels during the second half of 2020 as we continue to make investments in our technology and infrastructure. These actions, combined with our continued core deposit growth, should help drive further improvement in the productivity of our branch network and increase our deposits per branch. Looking ahead to 2020, we anticipate non-interest expenses between $42 million to $44 million per quarter.
Now, we'll take a look at asset quality. Our non-performing assets decreased to 87 basis points of total assets from 89 basis points at the end of the prior quarter, primarily due to an improved pace of resolution of non-performing loans. As of December 31, our non-performing assets included $4.2 million of government guaranteed loans. Excluding government guaranteed non-performing loans, our non-performing loans to total loans ratio was 89 basis points, down from 98 basis points at the end of the prior quarter. Our net charge-offs were $4 million in the quarter. Virtually, all of the net charge-offs during the quarter were attributed to the unguaranteed portion of US government guaranteed loans. Comparing year-over-year, net charge-offs remained flat at 37 basis points.
Our provision expense was $4.4 million, which covered charge-offs and resulted in an increase in our allowance for loan losses to 84 basis points of total loans and leases. Our coverage on non-performing loans, excluding the government guaranteed portion, was 95%. In addition to the traditional allowance as a percentage of loan and lease metric, we also analyze the allowance in conjunction with the acquisition accounting adjustments impacting our acquired portfolio. At December 31, the acquisition accounting adjustments, plus our allowance for loan and lease losses, represented 158 basis points of total loans and leases.
As you may have noticed in our prepared materials, we do not include the impact of CECL. As an emerging growth company, we intend to adopt the new accounting standard in 2023. As a result, we will not have an adjustment to our capital at this time, and we will continue to record accretion income in our earnings at a trajectory similar to prior quarters.
With that, I would like to pass the call back to Alberto.
Alberto J. Paracchini -- President & Chief Executive Officer
Thank you, Lindsay. I would like to wrap up today with a few comments about our outlook for 2020 and our priorities going forward. Before I do that, I'd like to quickly go back to last year at this time and look at our priorities for the coming year to see how well we did. In summary, we completed the integration and conversion of two banks, grew deposits nicely and continued pursuing disciplined loan growth. We also thought that we see opportunities in the market to add talent to the organization, and we feel we added some great people over the course of the year. Lastly, we wanted to continue to see improved profitability, and we're able to deliver on that as well.
For 2020, our strategy and areas of focus remain the same, continue to generate deposits by executing our relationship banking strategy, pursue disciplined loan growth and capitalize on opportunities with both customers and talent. In terms of loan growth, we expect growth for the year to be in the 6% to 8% range assuming some normalization in payoff activity and a stable rate environment. Given the additions we made in terms of staff that came in toward the second half of the year, we expect to see business from those hires becoming more pronounced in the second half of 2020. We will continue investing in infrastructure products and capabilities particularly in treasury management and continue to put technology in place to improve efficiencies and our customer experience. Given our market position, strong capital levels of liquidity, and most importantly, our team, we're optimistic about our ability to navigate the current environment and take advantage of opportunities in 2020.
That concludes our prepared remarks and now operator we can tell -- we can open up the call to questions.
Questions and Answers:
Operator
We will now begin the question-and-answer session. [Operator Instructions] The first question comes from Michael Perito from KBW. Please go ahead.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
Hey, good morning guys. Happy New Year.
Lindsay Y. Corby -- Chief Financial Officer
Happy New Year, Mike.
Alberto J. Paracchini -- President & Chief Executive Officer
Happy New Year, Mike.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
I had a few things I wanted to hit. I wanted to start on the expense side, the $42 million to $44 million kind of run rate for 2020, Lindsay, obviously it's a little wide. I'm curious how you think the trajectory of the year? I mean, it sounds like you're a little bit more optimistic, maybe about being toward the lower end in the back half of the year versus the first half. Is that kind of a fair read on your comments or would you got me in a different direction?
Lindsay Y. Corby -- Chief Financial Officer
That is a fair read on the comment. So we do have seasonality in our business, and we do tend to have higher expenses in the beginning of the year. So there is that level of seasonality that does take place. So I think that's fair, Mike.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
And then just kind of a follow-up on that same topic, as I think about kind of stability in the efficiency ratio that you guys have had over the last couple of years after the big improvement in 2018, what -- I realized it's heavily dependent on rates and margin. But if we just kind of assume your margin guidance at face value here and that rates don't move dramatically, I mean, what type of efficiency ratio do you guys think you could achieve in 2020? I mean, is it -- do you think you could continue to have the stability? Do you think there is some upward pressure? So any general thoughts on that topic.
Lindsay Y. Corby -- Chief Financial Officer
Sure. I think in terms of the efficiency, there are definitely headwinds. Obviously, you thought this quarter with the rate environment. So I think there is pressure here upfront, but I do think that in terms of the latter half of the year, we will see the ability to guide that down up into the high 50s.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
Okay. And then Alberto on capital, obviously you just have an active 2019 between Oak Park, dividend and share repurchases, you kind of mentioned it a little bit. I was wondering if you could expand maybe a little bit more for us just on what the capital plans are for 2020 and maybe specifically, kind of what the appetite is to use buybacks here as we move forward.
Alberto J. Paracchini -- President & Chief Executive Officer
Yeah. Mike, sure. Look, I think we certainly remain open to that. Obviously, that was one of the reasons why we instituted the program at the end of last year. I think flexibility is important. Obviously, if we look at our capital levels today, they are healthy, which is great. I mean, given the beta uncertainty still on the environment, I think that's prudent. That being said, we just recently instituted a dividend. So I think that's a good example of us taking action to return capital back to shareholders, and I would put the buyback in the context of capital priorities, first and foremost supporting organic growth. Then obviously, we want to have capital available when we do and see opportunities to potentially do M&A as we've done in the past but that being said, obviously the buyback as a tool to manage growing capital levels. I think we've accreted capital very nicely from, if you look at 2018 to 2019. So in the context of having the program in place, I certainly think that, that's a tool that we certainly have available to manage that and make sure that we're not creating capital, growing capital that we can't deploy in a reasonable amount of time.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
So I guess, from our perspective and I understand that it's hard question to answer because opportunities can shift, but from our perspective, as we try to model you guys out here, I mean is it fair to say that by this time next year, your hope is to use all the levers of your disposal, so that this 10.3% TCE ratio isn't really not much higher a year from now?
Alberto J. Paracchini -- President & Chief Executive Officer
I think that's fair, Mike. I think I would say obviously, the one caveat is opportunities that may present themselves over the course of the year that may change that position, but I think that's fair.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
And on that point, just any update on the deal pipeline?
Alberto J. Paracchini -- President & Chief Executive Officer
I think -- I would say -- I would -- we remain constructive on it. I think activity -- I would say if anything, maybe from over the course of the summer in terms of be it the pace or the conversations and discussions that we've had with parties, I think probably remain pretty healthy. I would tell you though probably seller expectations are probably a little high relative to reality today. That being said, I think in terms of activity and discussions and having opportunities to look at transactions, I think the environment remains pretty healthy.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
Great. And then just one last one for quick one from me and I'll step back. Lindsay, any thoughts on the tax rate for 2020?
Lindsay Y. Corby -- Chief Financial Officer
Sure. So that what you saw here in the fourth quarter was a one-time item. So our guidance going forward for the effective tax rate is between 26% to 28%, right.
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
Great. Thank you guys. Appreciate the color.
Alberto J. Paracchini -- President & Chief Executive Officer
Thanks, Mike.
Operator
[Operator Instructions] Next question comes from Nathan Race. Nathan. you may go.
Nathan Race -- Piper Jaffray and Company -- Analyst
Thank you. Good morning, everyone.
Lindsay Y. Corby -- Chief Financial Officer
Good morning, Nathan.
Nathan Race -- Piper Jaffray and Company -- Analyst
So maybe just start on deposit growth expectations, obviously pretty strong core deposit gathering in the fourth quarter. And so, just curious as we think about the core NIM outlook kind of hold and stable from here in 2020. How much opportunity exist to continue to grow core deposits at a similar clip than what we saw in the back half of last year to continue to kind of prune some higher cost deposit relationships and so forth?
Alberto J. Paracchini -- President & Chief Executive Officer
Yeah. Mike, I think I would answer that in the context of kind of loan growth and seeing deposits. We want to see deposits keep pace with that. It's -- we're primarily a relationship-oriented institution. So I would -- the guidance that I would give you is consistent with loan growth. We will obviously want to bank the full relationship and that's where core deposits come from. So to the degree that we continue to see the trends that we saw in the fourth quarter, I think you saw what we were able to do in terms of managing, call it, higher funding -- higher cost of fund type deposits down and replacing them with either non-interest bearing or other types of core account.
Nathan Race -- Piper Jaffray and Company -- Analyst
Understood. That's helpful. And then [Speech Overlap].
Alberto J. Paracchini -- President & Chief Executive Officer
Nathan, one thing that I want to add to that, obviously the other lever there, I made a comment related to increased liquidity given the dynamics in the portfolio in the fourth quarter. I would say, our investment portfolio today is probably a little higher in terms of just absolute level. So obviously, that's a level -- lever that we can utilize where we can redeploy some of that excess liquidity over time back into the loan portfolio. So I just wanted to add that context as well.
Nathan Race -- Piper Jaffray and Company -- Analyst
Understood. But it sounds like you kind of want to keep the loan deposit ratio near its current range around 91%, 92% over the course of this year.
Lindsay Y. Corby -- Chief Financial Officer
No. We think that we've got ample liquidity here that where we can -- we've got some runway here. So we've peaked, I think, at around a little over 95% and grew about 95% and change. And so, I really think by the end of 2020, we'd like to see that get back up to that range.
Nathan Race -- Piper Jaffray and Company -- Analyst
Okay, understood. Thank you. And then just changing gears a little bit on payoffs, we've heard from a couple of other Chicago banks so far this earnings season that they're seeing some moderation payoff activity thus far in 1Q. Are you seeing any of that as it relates to your portfolio?
Alberto J. Paracchini -- President & Chief Executive Officer
I think it's -- I mean I would say it's a call. I mean it's -- we're not even through the first month of the year. So I mean, I think a month is too early to tell, but that being said, let's say January has been -- definitely has been some moderation in the month of January. But I think the important thing from our perspective is 2019 as a whole was elevated. I think if you look at the chart on the presentation, you kind of saw pay. I was kind of increasing over the course of the year with the fourth quarter being higher.
A couple of things there, the rate environment and certainly the outlook for the rate environment changed materially midyear, hard to quantify the impact of that, but I think that had something to do particularly on the CRE side. The second thing is, for us, we had obviously portfolios that came in, particularly the acquisition of Oak Park River Forest Bankshares. There is always some transition there that happens. We saw some of that. So in terms of outlook for 2020, we do expect that we're going to see some moderation with a stable rate environment and some of the transition effects largely hopefully being done here by the end of the first quarter. I think payoff activity should moderate over the course of 2020.
Nathan Race -- Piper Jaffray and Company -- Analyst
Got it. It's great to hear. And then if I could just ask one more on SBA premiums came down sequentially in the fourth quarter and I would think short-term rates have kind of come down as well over the last several months that, that may have -- would have supported here again on sale premiums. Just any kind of visibility in terms of how things are trending in 1Q and just and so forth.
Lindsay Y. Corby -- Chief Financial Officer
Sure. So again, there does tend to be some volatility in that number. I think in the fourth quarter here, you saw the slight decrease was really just driven by the premiums. The volume was there in terms of what we sold. So we were happy about that fourth quarter. Looking into the next year, we look at it on an aggregate basis over the course of the year. However, in the first quarter -- for instance, if you look back at 2018, the first quarter does tend to be fairly light and so I'd say look at 2018 and that will give you a good idea in terms of how things flow over the course of the year.
Alberto J. Paracchini -- President & Chief Executive Officer
Yeah, Nat, to add to what Lindsay just said and this is just more on an aggregate basis when we look at kind of historical trends and premiums for SBA 7A only, but I think it's a fair statement. And when you look back relative to last quarter, Q3 compared to Q4 and certainly, before that, premiums, I think, it's fair to say are slightly down as a whole. This is gross premiums across the board, so kind of 10, 20 year -- 10, 15, 20, 25 years. I think it's fair to say premiums are slightly down over the last quarter as a whole.
Nathan Race -- Piper Jaffray and Company -- Analyst
Got it. Understood. I appreciate you guys taking the questions. Thank you.
Operator
The next question comes from Brian Martin from Janney Montgomery. Please go ahead.
Brian Martin -- Janney Montgomery Scott -- Analyst
Hey, good morning.
Lindsay Y. Corby -- Chief Financial Officer
Good morning.
Alberto J. Paracchini -- President & Chief Executive Officer
Hi, Brian.
Brian Martin -- Janney Montgomery Scott -- Analyst
So Lindsay, I guess you talked about or I don't know if Alberto just the kind of loan pricing stabilizing a bit. I mean it sounds like -- I appreciate the color on the funding side, still seeing some reductions here in the first half, but just what are you guys seeing on loan pricing kind of new production. It sounds like certainly it's competitive, but maybe stabilizing at current levels, giving you that optimism on the margin or am I hearing that wrong?
Alberto J. Paracchini -- President & Chief Executive Officer
No, I think -- look if we look at, for example, kind of new production and kind of where new production was at the end of the fourth quarter, I would tell you and sometimes, I'll give you this number. But just know that the mix of funded loans in a particular quarter can vary, but we saw kind of the weighted average rate be around 5.6% roughly speaking. If we compare that to the third quarter, kind of where we were, again, raw. This is not adjusting for mix changes from one quarter to the other. It was around 6.07%. So it's down around 40 basis points. That said, if you look at the change in one month LIBOR over the course of the third quarter relative to the fourth quarter, if you look -- depending [Phonetic] of the timing of that, you're going to see that LIBOR kind of shrunk by somewhere between 26 basis points to 30 basis points. So what we saw from a pricing standpoint, I think your comment -- I think it's relatively fair, but you can see how it's tied to movement in short-term rates that sometimes are not call it the Fed funds target. So hopefully, that gives you a sense of the variability there.
Brian Martin -- Janney Montgomery Scott -- Analyst
Yes, that's helped. I appreciate. And then maybe Lindsay, just a normal accretion just holistically, as we think about 2020, just kind of the percentage decline you would expect in the accretion, given that you're not going to be adopting CECL, just any thoughts on how we should model that.
Lindsay Y. Corby -- Chief Financial Officer
Sure. So again, it's never perfect to predict, Brian, as I've always stated in the past, but I do think that you'll see it continues to come down as I've always said in the past and it continues to stair-step down. I think you're going to see one more larger stair-step down here in the first quarter and then it tends to level out. Ideally, we can accelerate as much accretion as fast as we possibly can and resolve these loans with our credit team to get it through our earnings prior to CECL implementation. But from just the standpoint of where it's going, it will stair-step down here again, and then begin to moderate in the latter half of '20.
Brian Martin -- Janney Montgomery Scott -- Analyst
Okay. And then just one housekeeping on the fee income side, the loan servicing revaluation given kind of the outlook on rates today, any thought on if you start to see that decline or stabilize, I guess, any thoughts there?
Lindsay Y. Corby -- Chief Financial Officer
So I'd say on the servicing asset valuation, it does tend to be volatile and we really just had the perfect storm this quarter in terms of everything with the prepayment speeds and the discount rates. And so you saw more outsized valuation adjustment there than you would typically see. So if you look back historically, Brian and you average that, I think that's probably a pretty good fair assumption to look at.
Alberto J. Paracchini -- President & Chief Executive Officer
Yeah. I think just when you think about that, you think about two things. Obviously, you have your gross servicing fees, which is how we look at the business and then you have these fair value changes over time when you have to make, call it, adjustments to either the discount rate or prepayment speeds. If you take that in the context of the comments that we made regarding kind of SBA premiums being a little lower, you can use that as a kind of like a proxy in terms of what's happening with discount rate. So we saw this quarter discount rates in shop. So that's a fair value input change and then prepayment speeds, which is the reason why probably premiums are down a bit in the market as a whole, picking up that obviously had a fair value impact on the servicing asset. But from an operating basis, I mean what we tried to do is look at how is gross servicing income coming in, knowing that you're going to have for value adjustments like you do in mortgage MSRs from time-to-time.
Brian Martin -- Janney Montgomery Scott -- Analyst
Got you. Okay, that's helpful. And maybe just the last one, just in general, the buyback, I mean, I guess, are you guys looking at that today is more of a defensive mechanism where the stock were to drop. You could maybe get more assertive or do you just -- is it just kind of the normal plan. They have some of it going forward. Just any thoughts on that?
Alberto J. Paracchini -- President & Chief Executive Officer
Brian, I'll keep the comments to. It's a way to opportunistically manage our capital levels and to the question that Mike answer asked earlier in the call, I think that the point that he brought up, in the way he frame the question was well put, right. We obviously have -- we instituted the dividend. And I would look at both of those measures as the tools that we have in place to return capital and manage capital levels in the context of continuing to support organic growth and take advantage of acquisition opportunities when they surface.
Brian Martin -- Janney Montgomery Scott -- Analyst
Got you. Okay. I appreciate all the color. Thanks.
Lindsay Y. Corby -- Chief Financial Officer
Thanks, Brian.
Operator
There are no more questions in the queue. This concludes our question-and-answer session. I would like to turn the conference back over to management for any closing remarks.
Alberto J. Paracchini -- President & Chief Executive Officer
Thank you, operator. Before we close, I'd like to take this opportunity to say thank you to all of our colleagues for the contributions they make to our business on a daily basis. With all of the changes impacting our business today from a technology standpoint, banking is still a personal business and our colleagues are and will continue to be responsible for our success. I know a lot of our colleagues dial-in and listen to these calls. So for those of you on the call as well as all others who could not join today, I just want to say thank you again and look forward to another great year in 2020. That concludes the call for today. Thank you for your participation and your interest in Byline, and we'll talk to you again next quarter. Operator?
Operator
[Operator Closing Remarks]
Duration: 35 minutes
Call participants:
Alberto J. Paracchini -- President & Chief Executive Officer
Lindsay Y. Corby -- Chief Financial Officer
Tony Rossi -- Financial Profiles, Inc. -- Analyst
Michael Perito -- Keefe, Bruyette & Woods, Inc. -- Analyst
Nathan Race -- Piper Jaffray and Company -- Analyst
Brian Martin -- Janney Montgomery Scott -- Analyst
